Jul 14 11:18:24.071628 ip-10-0-133-87 systemd[1]: Starting Kubernetes Kubelet... Jul 14 11:18:24.526332 ip-10-0-133-87 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 11:18:24.526332 ip-10-0-133-87 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 14 11:18:24.526332 ip-10-0-133-87 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 11:18:24.526332 ip-10-0-133-87 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 14 11:18:24.526332 ip-10-0-133-87 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 11:18:24.527405 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.527307 2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 14 11:18:24.532498 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532483 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 11:18:24.532498 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532498 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 11:18:24.532561 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532501 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 11:18:24.532561 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532504 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 11:18:24.532561 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532509 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 11:18:24.532561 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532512 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 11:18:24.532655 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532565 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 11:18:24.532655 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532570 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 14 11:18:24.532655 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532574 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 11:18:24.532655 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532578 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 11:18:24.532655 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532582 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 11:18:24.532655 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532585 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 11:18:24.532655 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532588 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 11:18:24.532655 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532590 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 11:18:24.532655 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532593 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 11:18:24.532655 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532596 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 11:18:24.532655 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532599 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 11:18:24.532655 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532601 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 11:18:24.532655 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532604 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 11:18:24.532655 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532607 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 11:18:24.532655 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532609 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 11:18:24.532655 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532612 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 11:18:24.532655 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532615 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 11:18:24.532655 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532617 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 11:18:24.532655 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532620 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 11:18:24.532655 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532623 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 11:18:24.533126 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532625 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 11:18:24.533126 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532628 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 11:18:24.533126 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532631 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 11:18:24.533126 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532634 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 11:18:24.533126 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532636 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 11:18:24.533126 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532639 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 11:18:24.533126 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532642 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 11:18:24.533126 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532645 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 11:18:24.533126 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532647 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 11:18:24.533126 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532650 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 11:18:24.533126 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532652 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 11:18:24.533126 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532656 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 11:18:24.533126 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532658 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 11:18:24.533126 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532662 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 11:18:24.533126 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532667 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 11:18:24.533126 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532670 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 11:18:24.533126 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532673 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 11:18:24.533126 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532676 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 11:18:24.533126 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532680 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 11:18:24.533619 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532684 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 11:18:24.533619 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532687 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 11:18:24.533619 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532689 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 11:18:24.533619 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532692 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 11:18:24.533619 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532695 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 11:18:24.533619 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532697 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 11:18:24.533619 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532699 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 11:18:24.533619 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532702 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 11:18:24.533619 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532705 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 11:18:24.533619 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532708 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 11:18:24.533619 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532710 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 11:18:24.533619 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532712 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 11:18:24.533619 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532715 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 11:18:24.533619 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532717 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 11:18:24.533619 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532720 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 11:18:24.533619 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532732 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 11:18:24.533619 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532734 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 11:18:24.533619 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532737 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 11:18:24.533619 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532740 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 11:18:24.533619 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532742 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 11:18:24.534085 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532746 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 11:18:24.534085 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532748 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 11:18:24.534085 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532751 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 11:18:24.534085 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532754 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 11:18:24.534085 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532756 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 11:18:24.534085 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532759 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 11:18:24.534085 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532761 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 11:18:24.534085 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532764 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 11:18:24.534085 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532766 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 11:18:24.534085 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532769 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 11:18:24.534085 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532772 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 11:18:24.534085 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532774 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 11:18:24.534085 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532777 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 11:18:24.534085 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532779 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 11:18:24.534085 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532782 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 11:18:24.534085 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532784 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 11:18:24.534085 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532787 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 11:18:24.534085 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532789 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 11:18:24.534085 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532792 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 11:18:24.534085 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532795 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 11:18:24.534574 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532797 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 11:18:24.534574 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532801 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 11:18:24.534574 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532804 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 11:18:24.534574 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532806 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 11:18:24.534574 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532809 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 11:18:24.534574 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532811 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 11:18:24.534574 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532814 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 11:18:24.534574 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532819 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 11:18:24.534574 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532822 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 11:18:24.534574 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532825 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 11:18:24.534574 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532828 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 11:18:24.534574 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532831 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 11:18:24.534574 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532834 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 11:18:24.534574 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.532838 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 11:18:24.534934 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534707 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 11:18:24.534934 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534714 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 11:18:24.534934 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534717 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 11:18:24.534934 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534720 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 11:18:24.534934 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534723 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 11:18:24.534934 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534726 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 11:18:24.534934 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534728 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 11:18:24.534934 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534731 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 11:18:24.534934 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534733 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 11:18:24.534934 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534736 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 11:18:24.534934 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534740 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 11:18:24.534934 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534743 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 11:18:24.534934 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534746 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 11:18:24.534934 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534749 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 11:18:24.534934 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534752 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 11:18:24.534934 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534754 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 11:18:24.534934 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534757 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 11:18:24.534934 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534760 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 11:18:24.534934 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534762 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 11:18:24.535442 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534764 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 11:18:24.535442 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534767 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 11:18:24.535442 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534770 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 14 11:18:24.535442 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534772 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 11:18:24.535442 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534775 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 11:18:24.535442 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534777 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 11:18:24.535442 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534781 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 11:18:24.535442 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534784 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 11:18:24.535442 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534787 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 11:18:24.535442 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534790 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 11:18:24.535442 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534794 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 11:18:24.535442 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534796 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 11:18:24.535442 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534799 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 11:18:24.535442 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534803 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 11:18:24.535442 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534806 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 11:18:24.535442 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534809 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 11:18:24.535442 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534811 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 11:18:24.535442 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534814 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 11:18:24.535442 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534817 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 11:18:24.535442 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534819 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 11:18:24.535945 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534822 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 11:18:24.535945 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534824 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 11:18:24.535945 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534827 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 11:18:24.535945 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534829 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 11:18:24.535945 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534832 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 11:18:24.535945 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534834 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 11:18:24.535945 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534837 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 11:18:24.535945 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534840 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 11:18:24.535945 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534842 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 11:18:24.535945 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534845 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 11:18:24.535945 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534847 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 11:18:24.535945 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534850 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 11:18:24.535945 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534853 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 11:18:24.535945 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534855 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 11:18:24.535945 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534858 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 11:18:24.535945 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534860 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 11:18:24.535945 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534863 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 11:18:24.535945 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534866 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 11:18:24.535945 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534868 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 11:18:24.535945 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534871 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 11:18:24.536438 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534873 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 11:18:24.536438 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534876 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 11:18:24.536438 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534878 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 11:18:24.536438 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534882 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 11:18:24.536438 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534884 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 11:18:24.536438 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534887 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 11:18:24.536438 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534890 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 11:18:24.536438 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534893 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 11:18:24.536438 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534895 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 11:18:24.536438 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534898 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 11:18:24.536438 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534901 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 11:18:24.536438 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534903 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 11:18:24.536438 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534906 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 11:18:24.536438 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534908 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 11:18:24.536438 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534912 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 11:18:24.536438 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534914 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 11:18:24.536438 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534917 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 11:18:24.536438 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534919 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 11:18:24.536438 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534922 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 11:18:24.536438 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534924 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 11:18:24.536923 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534927 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 11:18:24.536923 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534929 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 11:18:24.536923 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534933 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 11:18:24.536923 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534936 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 11:18:24.536923 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534938 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 11:18:24.536923 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534941 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 11:18:24.536923 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534944 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 11:18:24.536923 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534947 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 11:18:24.536923 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534949 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 11:18:24.536923 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534952 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 11:18:24.536923 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534955 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 11:18:24.536923 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534957 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 11:18:24.536923 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534960 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 11:18:24.536923 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534962 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 11:18:24.536923 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534965 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 11:18:24.536923 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534968 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 11:18:24.536923 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534970 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 11:18:24.536923 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534973 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 11:18:24.536923 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534975 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 11:18:24.536923 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.534978 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 11:18:24.537406 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535338 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 11:18:24.537406 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535343 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 11:18:24.537406 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535364 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 11:18:24.537406 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535369 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 11:18:24.537406 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535372 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 11:18:24.537406 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535377 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 11:18:24.537406 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535381 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 11:18:24.537406 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535384 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 11:18:24.537406 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535386 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 11:18:24.537406 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535389 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 11:18:24.537406 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535391 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 11:18:24.537406 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535394 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 11:18:24.537406 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535396 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 11:18:24.537406 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535400 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 11:18:24.537406 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535402 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 11:18:24.537406 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535405 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 11:18:24.537406 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535408 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 11:18:24.537406 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535410 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 11:18:24.537406 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535413 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 11:18:24.537406 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535416 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 11:18:24.537895 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535418 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 11:18:24.537895 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535421 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 11:18:24.537895 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535423 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 11:18:24.537895 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535426 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 11:18:24.537895 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535428 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 11:18:24.537895 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535431 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 11:18:24.537895 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535434 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 11:18:24.537895 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535437 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 11:18:24.537895 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535439 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 11:18:24.537895 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535442 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 11:18:24.537895 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535444 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 11:18:24.537895 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535448 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 11:18:24.537895 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535451 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 11:18:24.537895 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535453 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 11:18:24.537895 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535456 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 11:18:24.537895 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535459 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 11:18:24.537895 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535462 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 11:18:24.537895 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535481 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 11:18:24.537895 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535484 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 11:18:24.537895 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535488 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 11:18:24.538379 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535491 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 11:18:24.538379 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535494 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 11:18:24.538379 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535497 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 11:18:24.538379 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535499 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 11:18:24.538379 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535502 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 11:18:24.538379 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535505 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 11:18:24.538379 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535507 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 11:18:24.538379 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535511 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 11:18:24.538379 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535515 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 11:18:24.538379 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535519 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 11:18:24.538379 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535521 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 11:18:24.538379 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535524 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 11:18:24.538379 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535526 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 11:18:24.538379 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535529 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 11:18:24.538379 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535532 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 11:18:24.538379 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535535 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 11:18:24.538379 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535537 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 11:18:24.538379 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535540 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 11:18:24.538379 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535543 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 11:18:24.538823 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535545 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 11:18:24.538823 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535548 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 11:18:24.538823 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535551 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 11:18:24.538823 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535553 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 11:18:24.538823 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535555 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 11:18:24.538823 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535558 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 11:18:24.538823 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535561 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 14 11:18:24.538823 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535563 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 11:18:24.538823 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535568 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 11:18:24.538823 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535571 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 11:18:24.538823 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535573 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 11:18:24.538823 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535576 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 11:18:24.538823 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535578 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 11:18:24.538823 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535580 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 11:18:24.538823 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535583 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 11:18:24.538823 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535586 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 11:18:24.538823 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535588 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 11:18:24.538823 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535592 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 11:18:24.538823 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535595 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 11:18:24.538823 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535597 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 11:18:24.539292 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535599 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 11:18:24.539292 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535602 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 11:18:24.539292 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535604 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 11:18:24.539292 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535607 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 11:18:24.539292 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535609 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 11:18:24.539292 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535612 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 11:18:24.539292 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535615 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 11:18:24.539292 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535619 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 11:18:24.539292 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535622 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 11:18:24.539292 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535624 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 11:18:24.539292 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535627 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 11:18:24.539292 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535632 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 11:18:24.539292 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535635 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 11:18:24.539292 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535638 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 11:18:24.539292 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535640 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 11:18:24.539292 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535643 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 11:18:24.539292 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535647 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 11:18:24.539292 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535650 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 11:18:24.539292 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535654 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.535657 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535718 2571 flags.go:64] FLAG: --address="0.0.0.0" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535725 2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535732 2571 flags.go:64] FLAG: --anonymous-auth="true" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535736 2571 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535740 2571 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535743 2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535757 2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535762 2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535765 2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535768 2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535771 2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535774 2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535778 2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535781 2571 flags.go:64] FLAG: --cgroup-root="" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535784 2571 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535787 2571 flags.go:64] FLAG: --client-ca-file="" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535790 2571 flags.go:64] FLAG: --cloud-provider="external" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535793 2571 flags.go:64] FLAG: --cluster-dns="[]" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535797 2571 flags.go:64] FLAG: --cluster-domain="" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535800 2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535803 2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535806 2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 14 11:18:24.539758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535809 2571 flags.go:64] FLAG: --container-log-max-files="5" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535814 2571 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535817 2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535820 2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535823 2571 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535827 2571 flags.go:64] FLAG: --contention-profiling="false" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535829 2571 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535832 2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535836 2571 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535839 2571 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535844 2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535848 2571 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535850 2571 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535853 2571 flags.go:64] FLAG: --enable-load-reader="false" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535856 2571 flags.go:64] FLAG: --enable-server="true" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535859 2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535864 2571 flags.go:64] FLAG: --event-burst="100" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535868 2571 flags.go:64] FLAG: --event-qps="50" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535870 2571 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535873 2571 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535876 2571 flags.go:64] FLAG: --eviction-hard="" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535880 2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535883 2571 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535886 2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535889 2571 flags.go:64] FLAG: --eviction-soft="" Jul 14 11:18:24.540336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535891 2571 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535894 2571 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535897 2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535900 2571 flags.go:64] FLAG: --experimental-mounter-path="" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535902 2571 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535905 2571 flags.go:64] FLAG: --fail-swap-on="true" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535908 2571 flags.go:64] FLAG: --feature-gates="" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535911 2571 flags.go:64] FLAG: --file-check-frequency="20s" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535915 2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535918 2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535921 2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535925 2571 flags.go:64] FLAG: --healthz-port="10248" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535928 2571 flags.go:64] FLAG: --help="false" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535931 2571 flags.go:64] FLAG: --hostname-override="ip-10-0-133-87.ec2.internal" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535934 2571 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535937 2571 flags.go:64] FLAG: --http-check-frequency="20s" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535940 2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535946 2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535949 2571 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535952 2571 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535955 2571 flags.go:64] FLAG: --image-service-endpoint="" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535958 2571 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535961 2571 flags.go:64] FLAG: --kube-api-burst="100" Jul 14 11:18:24.541003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535964 2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535967 2571 flags.go:64] FLAG: --kube-api-qps="50" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535970 2571 flags.go:64] FLAG: --kube-reserved="" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535973 2571 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535976 2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535979 2571 flags.go:64] FLAG: --kubelet-cgroups="" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535982 2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535984 2571 flags.go:64] FLAG: --lock-file="" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535987 2571 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535990 2571 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535993 2571 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.535998 2571 flags.go:64] FLAG: --log-json-split-stream="false" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536001 2571 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536005 2571 flags.go:64] FLAG: --log-text-split-stream="false" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536007 2571 flags.go:64] FLAG: --logging-format="text" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536010 2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536014 2571 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536017 2571 flags.go:64] FLAG: --manifest-url="" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536020 2571 flags.go:64] FLAG: --manifest-url-header="" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536024 2571 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536027 2571 flags.go:64] FLAG: --max-open-files="1000000" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536031 2571 flags.go:64] FLAG: --max-pods="110" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536034 2571 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536037 2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536040 2571 flags.go:64] FLAG: --memory-manager-policy="None" Jul 14 11:18:24.541633 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536043 2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536047 2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536051 2571 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536054 2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536061 2571 flags.go:64] FLAG: --node-status-max-images="50" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536064 2571 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536067 2571 flags.go:64] FLAG: --oom-score-adj="-999" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536070 2571 flags.go:64] FLAG: --pod-cidr="" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536073 2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536078 2571 flags.go:64] FLAG: --pod-manifest-path="" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536081 2571 flags.go:64] FLAG: --pod-max-pids="-1" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536084 2571 flags.go:64] FLAG: --pods-per-core="0" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536087 2571 flags.go:64] FLAG: --port="10250" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536090 2571 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536093 2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01f1af78592aa998b" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536096 2571 flags.go:64] FLAG: --qos-reserved="" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536099 2571 flags.go:64] FLAG: --read-only-port="10255" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536102 2571 flags.go:64] FLAG: --register-node="true" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536105 2571 flags.go:64] FLAG: --register-with-taints="" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536108 2571 flags.go:64] FLAG: --registry-burst="10" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536111 2571 flags.go:64] FLAG: --registry-qps="5" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536114 2571 flags.go:64] FLAG: --reserved-cpus="" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536117 2571 flags.go:64] FLAG: --reserved-memory="" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536120 2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536123 2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 14 11:18:24.542230 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536127 2571 flags.go:64] FLAG: --rotate-certificates="false" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536130 2571 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536133 2571 flags.go:64] FLAG: --runonce="false" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536136 2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536139 2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536142 2571 flags.go:64] FLAG: --seccomp-default="false" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536145 2571 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536147 2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536150 2571 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536153 2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536157 2571 flags.go:64] FLAG: --storage-driver-password="root" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536160 2571 flags.go:64] FLAG: --storage-driver-secure="false" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536162 2571 flags.go:64] FLAG: --storage-driver-table="stats" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536165 2571 flags.go:64] FLAG: --storage-driver-user="root" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536168 2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536171 2571 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536174 2571 flags.go:64] FLAG: --system-cgroups="" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536177 2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536182 2571 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536184 2571 flags.go:64] FLAG: --tls-cert-file="" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536187 2571 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536191 2571 flags.go:64] FLAG: --tls-min-version="" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536194 2571 flags.go:64] FLAG: --tls-private-key-file="" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536197 2571 flags.go:64] FLAG: --topology-manager-policy="none" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536199 2571 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 14 11:18:24.542861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536202 2571 flags.go:64] FLAG: --topology-manager-scope="container" Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536205 2571 flags.go:64] FLAG: --v="2" Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536208 2571 flags.go:64] FLAG: --version="false" Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536212 2571 flags.go:64] FLAG: --vmodule="" Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536216 2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.536219 2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536335 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536340 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536343 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536365 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536368 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536371 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536374 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536377 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536380 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536382 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536385 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536387 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536391 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536394 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536397 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536399 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 11:18:24.543489 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536402 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 11:18:24.544410 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536405 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 11:18:24.544410 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536407 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 11:18:24.544410 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536410 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 11:18:24.544410 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536413 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 11:18:24.544410 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536415 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 11:18:24.544410 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536418 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 11:18:24.544410 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536420 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 11:18:24.544410 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536425 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 11:18:24.544410 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536428 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 11:18:24.544410 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536431 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 11:18:24.544410 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536434 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 11:18:24.544410 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536437 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 11:18:24.544410 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536440 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 11:18:24.544410 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536442 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 11:18:24.544410 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536445 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 11:18:24.544410 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536448 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 11:18:24.544410 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536454 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 11:18:24.544410 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536457 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 11:18:24.544410 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536460 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 11:18:24.544984 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536464 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 11:18:24.544984 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536467 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 11:18:24.544984 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536471 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 11:18:24.544984 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536474 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 11:18:24.544984 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536477 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 11:18:24.544984 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536480 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 11:18:24.544984 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536483 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 11:18:24.544984 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536485 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 14 11:18:24.544984 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536488 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 11:18:24.544984 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536491 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 11:18:24.544984 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536494 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 11:18:24.544984 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536496 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 11:18:24.544984 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536499 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 11:18:24.544984 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536501 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 11:18:24.544984 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536504 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 11:18:24.544984 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536506 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 11:18:24.544984 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536509 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 11:18:24.544984 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536511 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 11:18:24.544984 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536514 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 11:18:24.545453 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536516 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 11:18:24.545453 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536519 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 11:18:24.545453 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536522 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 11:18:24.545453 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536524 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 11:18:24.545453 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536527 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 11:18:24.545453 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536529 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 11:18:24.545453 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536532 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 11:18:24.545453 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536534 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 11:18:24.545453 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536537 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 11:18:24.545453 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536541 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 11:18:24.545453 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536546 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 11:18:24.545453 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536549 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 11:18:24.545453 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536551 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 11:18:24.545453 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536554 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 11:18:24.545453 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536557 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 11:18:24.545453 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536559 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 11:18:24.545453 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536562 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 11:18:24.545453 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536564 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 11:18:24.545453 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536567 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 11:18:24.545453 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536569 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 11:18:24.545947 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536572 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 11:18:24.545947 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536575 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 11:18:24.545947 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536577 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 11:18:24.545947 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536580 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 11:18:24.545947 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536583 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 11:18:24.545947 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536585 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 11:18:24.545947 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536588 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 11:18:24.545947 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536590 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 11:18:24.545947 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536592 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 11:18:24.545947 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536595 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 11:18:24.545947 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536597 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 11:18:24.545947 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536600 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 11:18:24.545947 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536603 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 11:18:24.545947 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536605 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 11:18:24.545947 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536608 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 11:18:24.545947 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536610 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 11:18:24.545947 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536612 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 11:18:24.545947 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536615 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 11:18:24.545947 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536617 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 11:18:24.546420 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536620 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 11:18:24.546420 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536622 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 11:18:24.546420 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536626 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 11:18:24.546420 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536630 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 11:18:24.546420 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.536633 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 11:18:24.546420 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.537292 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 11:18:24.546420 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.546261 2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 14 11:18:24.546420 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.546276 2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 14 11:18:24.548579 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.548567 2571 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 14 11:18:24.548613 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.548579 2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 14 11:18:24.548649 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548644 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 11:18:24.548680 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548649 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 11:18:24.548680 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548653 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 11:18:24.548680 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548656 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 11:18:24.548680 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548658 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 11:18:24.548680 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548661 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 11:18:24.548680 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548664 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 11:18:24.548680 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548666 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 11:18:24.548680 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548669 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 11:18:24.548680 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548671 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 11:18:24.548680 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548674 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 11:18:24.548680 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548677 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 11:18:24.548680 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548679 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 11:18:24.548680 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548682 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 11:18:24.548680 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548685 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 11:18:24.549022 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548688 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 14 11:18:24.549022 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548691 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 11:18:24.549022 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548693 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 11:18:24.549022 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548696 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 11:18:24.549022 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548700 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 11:18:24.549022 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548704 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 11:18:24.549022 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548707 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 11:18:24.549022 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548710 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 11:18:24.549022 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548712 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 11:18:24.549022 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548716 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 11:18:24.549022 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548718 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 11:18:24.549022 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548729 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 11:18:24.549022 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548732 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 11:18:24.549022 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548740 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 11:18:24.549022 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548743 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 11:18:24.549022 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548745 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 11:18:24.549022 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548748 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 11:18:24.549022 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548750 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 11:18:24.549022 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548753 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 11:18:24.549022 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548756 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 11:18:24.549599 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548758 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 11:18:24.549599 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548761 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 11:18:24.549599 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548763 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 11:18:24.549599 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548766 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 11:18:24.549599 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548768 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 11:18:24.549599 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548771 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 11:18:24.549599 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548773 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 11:18:24.549599 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548776 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 11:18:24.549599 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548778 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 11:18:24.549599 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548781 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 11:18:24.549599 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548785 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 11:18:24.549599 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548788 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 11:18:24.549599 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548791 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 11:18:24.549599 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548794 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 11:18:24.549599 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548797 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 11:18:24.549599 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548801 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 11:18:24.549599 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548805 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 11:18:24.549599 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548808 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 11:18:24.549599 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548810 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 11:18:24.550043 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548813 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 11:18:24.550043 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548815 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 11:18:24.550043 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548818 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 11:18:24.550043 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548821 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 11:18:24.550043 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548823 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 11:18:24.550043 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548826 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 11:18:24.550043 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548829 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 11:18:24.550043 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548831 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 11:18:24.550043 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548834 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 11:18:24.550043 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548837 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 11:18:24.550043 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548840 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 11:18:24.550043 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548842 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 11:18:24.550043 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548845 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 11:18:24.550043 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548848 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 11:18:24.550043 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548851 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 11:18:24.550043 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548853 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 11:18:24.550043 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548855 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 11:18:24.550043 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548858 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 11:18:24.550043 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548860 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 11:18:24.550514 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548863 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 11:18:24.550514 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548866 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 11:18:24.550514 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548869 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 11:18:24.550514 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548871 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 11:18:24.550514 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548874 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 11:18:24.550514 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548877 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 11:18:24.550514 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548880 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 11:18:24.550514 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548883 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 11:18:24.550514 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548886 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 11:18:24.550514 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548888 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 11:18:24.550514 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548891 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 11:18:24.550514 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548894 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 11:18:24.550514 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548896 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 11:18:24.550514 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548899 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 11:18:24.550514 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548901 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 11:18:24.550514 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548904 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 11:18:24.550514 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548906 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 11:18:24.550514 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548909 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 11:18:24.550514 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548911 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 11:18:24.550514 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548913 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 11:18:24.550993 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548916 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 11:18:24.550993 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548919 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 11:18:24.550993 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548922 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 11:18:24.550993 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548925 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 11:18:24.550993 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548927 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 11:18:24.550993 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.548930 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 11:18:24.550993 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.548935 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 11:18:24.550993 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549059 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 11:18:24.550993 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549065 2571 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 11:18:24.550993 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549068 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 11:18:24.550993 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549071 2571 feature_gate.go:328] unrecognized feature gate: Example Jul 14 11:18:24.550993 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549074 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 11:18:24.550993 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549077 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 11:18:24.550993 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549080 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 11:18:24.550993 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549083 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 11:18:24.551383 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549085 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 11:18:24.551383 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549088 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 11:18:24.551383 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549090 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 11:18:24.551383 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549092 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 11:18:24.551383 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549095 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 11:18:24.551383 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549098 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 11:18:24.551383 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549101 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 11:18:24.551383 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549104 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 11:18:24.551383 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549106 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 11:18:24.551383 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549109 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 11:18:24.551383 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549111 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 11:18:24.551383 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549114 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 11:18:24.551383 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549116 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 11:18:24.551383 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549119 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 11:18:24.551383 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549122 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 11:18:24.551383 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549125 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 11:18:24.551383 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549127 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 11:18:24.551383 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549130 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 11:18:24.551383 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549132 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 11:18:24.551383 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549135 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 11:18:24.551855 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549137 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 11:18:24.551855 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549140 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 11:18:24.551855 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549143 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 11:18:24.551855 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549145 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 11:18:24.551855 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549148 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 11:18:24.551855 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549150 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 11:18:24.551855 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549153 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 11:18:24.551855 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549155 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 11:18:24.551855 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549158 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 11:18:24.551855 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549160 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 11:18:24.551855 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549163 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 11:18:24.551855 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549165 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 11:18:24.551855 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549168 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 11:18:24.551855 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549170 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 11:18:24.551855 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549172 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 11:18:24.551855 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549175 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 11:18:24.551855 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549177 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 11:18:24.551855 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549181 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 11:18:24.551855 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549184 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 11:18:24.551855 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549186 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 11:18:24.552329 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549188 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 11:18:24.552329 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549191 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 11:18:24.552329 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549193 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 11:18:24.552329 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549196 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 11:18:24.552329 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549198 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 11:18:24.552329 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549201 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 11:18:24.552329 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549203 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 11:18:24.552329 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549206 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 11:18:24.552329 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549208 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 11:18:24.552329 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549211 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 11:18:24.552329 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549213 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 11:18:24.552329 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549216 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 11:18:24.552329 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549219 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 11:18:24.552329 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549221 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 11:18:24.552329 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549224 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 11:18:24.552329 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549227 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 11:18:24.552329 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549230 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 11:18:24.552329 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549234 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 11:18:24.552329 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549237 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 11:18:24.552329 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549240 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 11:18:24.552838 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549243 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 11:18:24.552838 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549245 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 11:18:24.552838 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549248 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 11:18:24.552838 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549250 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 11:18:24.552838 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549254 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 11:18:24.552838 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549257 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 11:18:24.552838 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549260 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 11:18:24.552838 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549263 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 11:18:24.552838 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549266 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 11:18:24.552838 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549269 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 11:18:24.552838 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549272 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 11:18:24.552838 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549275 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 11:18:24.552838 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549277 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 11:18:24.552838 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549280 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 11:18:24.552838 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549282 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 11:18:24.552838 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549285 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 11:18:24.552838 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549287 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 11:18:24.552838 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549290 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 11:18:24.552838 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549292 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 11:18:24.552838 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549295 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 11:18:24.553318 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549297 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 11:18:24.553318 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549300 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 11:18:24.553318 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549302 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 11:18:24.553318 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549304 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 11:18:24.553318 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549307 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 11:18:24.553318 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549310 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 11:18:24.553318 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549312 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 11:18:24.553318 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549315 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 11:18:24.553318 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549317 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 11:18:24.553318 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549319 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 11:18:24.553318 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:24.549322 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 11:18:24.553318 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.549327 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 11:18:24.553318 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.549454 2571 server.go:962] "Client rotation is on, will bootstrap in background" Jul 14 11:18:24.553318 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.551550 2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 14 11:18:24.555845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.555832 2571 server.go:1019] "Starting client certificate rotation" Jul 14 11:18:24.555951 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.555933 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 11:18:24.555984 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.555966 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 11:18:24.558228 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.558215 2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 11:18:24.559532 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.559513 2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 11:18:24.569443 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.569423 2571 log.go:25] "Validated CRI v1 runtime API" Jul 14 11:18:24.571615 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.571598 2571 log.go:25] "Validated CRI v1 image API" Jul 14 11:18:24.573512 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.573496 2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 14 11:18:24.576270 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.576245 2571 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9fcacabf-74ef-49f0-be6e-e14105d85735:/dev/nvme0n1p4 f3482459-6268-417b-b349-bbe26b4317ff:/dev/nvme0n1p3] Jul 14 11:18:24.576338 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.576269 2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 14 11:18:24.583174 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.583031 2571 manager.go:216] Machine: {Timestamp:2026-07-14 11:18:24.581826796 +0000 UTC m=+0.375184427 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099093 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24f8cca70f7a54d6f91d28cddd5a77 SystemUUID:ec24f8cc-a70f-7a54-d6f9-1d28cddd5a77 BootID:07e07fbe-f38a-4643-8232-3e699195b61e Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:73:d9:33:1c:33 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:73:d9:33:1c:33 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:9b:9e:ae:37:f8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 14 11:18:24.583174 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.583163 2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 14 11:18:24.583308 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.583241 2571 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 14 11:18:24.586691 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.586675 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 11:18:24.586970 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.586947 2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 14 11:18:24.587100 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.586975 2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-87.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 14 11:18:24.587716 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.587706 2571 topology_manager.go:138] "Creating topology manager with none policy" Jul 14 11:18:24.587754 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.587718 2571 container_manager_linux.go:309] "Creating device plugin manager" Jul 14 11:18:24.587754 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.587725 2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 11:18:24.588335 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.588326 2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 11:18:24.589180 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.589169 2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 14 11:18:24.592472 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.592462 2571 state_mem.go:36] "Initialized new in-memory state store" Jul 14 11:18:24.592591 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.592582 2571 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 14 11:18:24.592620 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.592610 2571 kubelet.go:486] "Attempting to sync node with API server" Jul 14 11:18:24.592651 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.592621 2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 14 11:18:24.592651 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.592635 2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 14 11:18:24.592651 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.592644 2571 kubelet.go:398] "Adding apiserver pod source" Jul 14 11:18:24.592739 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.592653 2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 14 11:18:24.593019 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.593008 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 11:18:24.593070 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.593032 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 11:18:24.595691 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.595676 2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 14 11:18:24.597162 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.597148 2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 14 11:18:24.597215 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.597184 2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 14 11:18:24.597958 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.597946 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 14 11:18:24.598008 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.597966 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 14 11:18:24.598008 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.597976 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 14 11:18:24.598008 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.597983 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 14 11:18:24.598008 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.597994 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 14 11:18:24.598008 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.598001 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 14 11:18:24.599812 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.599798 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 14 11:18:24.599812 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.599811 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 14 11:18:24.599895 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.599819 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 14 11:18:24.599895 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.599825 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 14 11:18:24.599895 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.599834 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 14 11:18:24.599895 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.599841 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 14 11:18:24.602643 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.602629 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 14 11:18:24.602643 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.602644 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 14 11:18:24.604159 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:24.604126 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-87.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 14 11:18:24.604213 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:24.604120 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 14 11:18:24.605794 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.605781 2571 server.go:1268] "Started kubelet" Jul 14 11:18:24.606065 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.606027 2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 14 11:18:24.606134 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.606066 2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 14 11:18:24.606191 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.606152 2571 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 14 11:18:24.606579 ip-10-0-133-87 systemd[1]: Started Kubernetes Kubelet. Jul 14 11:18:24.607585 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.607566 2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 14 11:18:24.607665 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.607583 2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-87.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 14 11:18:24.608951 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.608933 2571 server.go:310] "Adding debug handlers to kubelet server" Jul 14 11:18:24.610994 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.610973 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wmtk9" Jul 14 11:18:24.611100 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:24.610018 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-87.ec2.internal.18c223ed53691afe default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-87.ec2.internal,UID:ip-10-0-133-87.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-87.ec2.internal,},FirstTimestamp:2026-07-14 11:18:24.605756158 +0000 UTC m=+0.399113790,LastTimestamp:2026-07-14 11:18:24.605756158 +0000 UTC m=+0.399113790,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-87.ec2.internal,}" Jul 14 11:18:24.611626 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:24.611611 2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 14 11:18:24.612226 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.612208 2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 14 11:18:24.612226 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.612214 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 14 11:18:24.612432 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:24.612414 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 14 11:18:24.613867 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.613736 2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 14 11:18:24.613942 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.613749 2571 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 14 11:18:24.613942 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.613917 2571 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 14 11:18:24.614112 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.614097 2571 reconstruct.go:97] "Volume reconstruction finished" Jul 14 11:18:24.614112 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.614112 2571 reconciler.go:29] "Reconciler: start to sync state" Jul 14 11:18:24.614945 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.614927 2571 factory.go:153] Registering CRI-O factory Jul 14 11:18:24.615025 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.614958 2571 factory.go:223] Registration of the crio container factory successfully Jul 14 11:18:24.615092 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.615030 2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 14 11:18:24.615092 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.615040 2571 factory.go:55] Registering systemd factory Jul 14 11:18:24.615092 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.615048 2571 factory.go:223] Registration of the systemd container factory successfully Jul 14 11:18:24.615092 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.615069 2571 factory.go:103] Registering Raw factory Jul 14 11:18:24.615092 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.615082 2571 manager.go:1196] Started watching for new ooms in manager Jul 14 11:18:24.615734 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.615718 2571 manager.go:349] Starting recovery of all containers Jul 14 11:18:24.621609 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:24.621579 2571 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-87.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 14 11:18:24.621719 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:24.621660 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 14 11:18:24.621813 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:24.621793 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wmtk9" Jul 14 11:18:26.260068 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.259909 2571 manager.go:354] Recovery completed Jul 14 11:18:26.264120 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.264107 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:18:26.266021 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.266008 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:18:26.266083 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.266037 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:18:26.266083 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.266052 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:18:26.266594 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.266576 2571 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 14 11:18:26.266594 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.266594 2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 14 11:18:26.266714 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.266597 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:18:26.266714 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.266612 2571 state_mem.go:36] "Initialized new in-memory state store" Jul 14 11:18:26.267294 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.267280 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:18:26.267343 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.267305 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:18:26.267380 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.267355 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:18:26.267882 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.267868 2571 policy_none.go:49] "None policy: Start" Jul 14 11:18:26.267956 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.267884 2571 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 14 11:18:26.267956 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.267894 2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 14 11:18:26.268695 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:26.268565 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-87.ec2.internal.18c223edb65ebd44 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-87.ec2.internal,UID:ip-10-0-133-87.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-87.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-87.ec2.internal,},FirstTimestamp:2026-07-14 11:18:26.266021188 +0000 UTC m=+0.427097649,LastTimestamp:2026-07-14 11:18:26.266021188 +0000 UTC m=+0.427097649,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-87.ec2.internal,}" Jul 14 11:18:26.269078 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.269063 2571 policy_none.go:47] "Start" Jul 14 11:18:26.305195 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.305174 2571 manager.go:337] "Starting Device Plugin manager" Jul 14 11:18:26.305300 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:26.305211 2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 14 11:18:26.305300 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.305220 2571 server.go:87] "Starting device plugin registration server" Jul 14 11:18:26.305480 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.305467 2571 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 14 11:18:26.305539 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.305482 2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 14 11:18:26.305615 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.305591 2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 14 11:18:26.305718 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.305701 2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 14 11:18:26.305718 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.305715 2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 14 11:18:26.306239 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:26.306218 2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 14 11:18:26.306317 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:26.306272 2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-87.ec2.internal\" not found" Jul 14 11:18:26.395794 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.395758 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 14 11:18:26.397151 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.396993 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 14 11:18:26.397151 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.397033 2571 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 14 11:18:26.397151 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.397055 2571 kubelet.go:2448] "Starting kubelet main sync loop" Jul 14 11:18:26.397151 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:26.397087 2571 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 14 11:18:26.399126 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.399105 2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 11:18:26.405728 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.405713 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:18:26.407016 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.407000 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:18:26.407094 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.407031 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:18:26.407094 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.407043 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:18:26.407094 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.407064 2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-87.ec2.internal" Jul 14 11:18:26.412746 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.412698 2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-87.ec2.internal" Jul 14 11:18:26.412746 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:26.412718 2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-87.ec2.internal\": node \"ip-10-0-133-87.ec2.internal\" not found" Jul 14 11:18:26.425874 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:26.425850 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 14 11:18:26.497595 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.497561 2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal"] Jul 14 11:18:26.497734 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.497643 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:18:26.501023 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.500998 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:18:26.501111 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.501038 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:18:26.501111 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.501050 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:18:26.502441 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.502428 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:18:26.502560 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.502546 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" Jul 14 11:18:26.502616 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.502585 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:18:26.503116 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.503087 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:18:26.503116 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.503088 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:18:26.503116 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.503117 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:18:26.503275 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.503129 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:18:26.503275 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.503132 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:18:26.503275 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.503145 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:18:26.504351 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.504339 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal" Jul 14 11:18:26.504418 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.504361 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 11:18:26.504939 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.504923 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientMemory" Jul 14 11:18:26.505020 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.504950 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 11:18:26.505020 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.504965 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientPID" Jul 14 11:18:26.520246 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:26.520224 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-87.ec2.internal\" not found" node="ip-10-0-133-87.ec2.internal" Jul 14 11:18:26.526020 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:26.525996 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 14 11:18:26.526117 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:26.526101 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-87.ec2.internal\" not found" node="ip-10-0-133-87.ec2.internal" Jul 14 11:18:26.547422 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.547399 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a8516bf7cc7b9f5f8ba46ac474d2b788-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal\" (UID: \"a8516bf7cc7b9f5f8ba46ac474d2b788\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" Jul 14 11:18:26.547502 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.547424 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8516bf7cc7b9f5f8ba46ac474d2b788-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal\" (UID: \"a8516bf7cc7b9f5f8ba46ac474d2b788\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" Jul 14 11:18:26.547502 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.547440 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/90fe358c1daa74a54db164d6ac35b548-config\") pod \"kube-apiserver-proxy-ip-10-0-133-87.ec2.internal\" (UID: \"90fe358c1daa74a54db164d6ac35b548\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal" Jul 14 11:18:26.626343 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:26.626311 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 14 11:18:26.647666 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.647617 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a8516bf7cc7b9f5f8ba46ac474d2b788-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal\" (UID: \"a8516bf7cc7b9f5f8ba46ac474d2b788\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" Jul 14 11:18:26.647764 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.647675 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8516bf7cc7b9f5f8ba46ac474d2b788-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal\" (UID: \"a8516bf7cc7b9f5f8ba46ac474d2b788\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" Jul 14 11:18:26.647764 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.647699 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/90fe358c1daa74a54db164d6ac35b548-config\") pod \"kube-apiserver-proxy-ip-10-0-133-87.ec2.internal\" (UID: \"90fe358c1daa74a54db164d6ac35b548\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal" Jul 14 11:18:26.647764 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.647729 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a8516bf7cc7b9f5f8ba46ac474d2b788-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal\" (UID: \"a8516bf7cc7b9f5f8ba46ac474d2b788\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" Jul 14 11:18:26.647878 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.647733 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8516bf7cc7b9f5f8ba46ac474d2b788-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal\" (UID: \"a8516bf7cc7b9f5f8ba46ac474d2b788\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" Jul 14 11:18:26.647878 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.647740 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/90fe358c1daa74a54db164d6ac35b548-config\") pod \"kube-apiserver-proxy-ip-10-0-133-87.ec2.internal\" (UID: \"90fe358c1daa74a54db164d6ac35b548\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal" Jul 14 11:18:26.727004 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:26.726952 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 14 11:18:26.824513 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.824488 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" Jul 14 11:18:26.827089 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:26.827062 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 14 11:18:26.829623 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:26.829608 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal" Jul 14 11:18:26.928015 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:26.927977 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 14 11:18:27.028592 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:27.028515 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 14 11:18:27.129068 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:27.129042 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 14 11:18:27.187572 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:27.187550 2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 14 11:18:27.188066 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:27.187699 2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 11:18:27.229998 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:27.229967 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 14 11:18:27.258973 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:27.258945 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 11:13:26 +0000 UTC" deadline="2028-03-13 15:31:45.870425648 +0000 UTC" Jul 14 11:18:27.258973 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:27.258969 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14596h13m18.611459012s" Jul 14 11:18:27.330499 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:27.330471 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 14 11:18:27.383266 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:27.383233 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8516bf7cc7b9f5f8ba46ac474d2b788.slice/crio-4db3ac0c1402f15cf9518ef4a5f29571a433073c4b3e34558fe5c2292ac2c949 WatchSource:0}: Error finding container 4db3ac0c1402f15cf9518ef4a5f29571a433073c4b3e34558fe5c2292ac2c949: Status 404 returned error can't find the container with id 4db3ac0c1402f15cf9518ef4a5f29571a433073c4b3e34558fe5c2292ac2c949 Jul 14 11:18:27.385423 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:27.385394 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90fe358c1daa74a54db164d6ac35b548.slice/crio-73812435990f1f46454dc2f07262def2dc4af98607890f65f960f6a01f27ae6f WatchSource:0}: Error finding container 73812435990f1f46454dc2f07262def2dc4af98607890f65f960f6a01f27ae6f: Status 404 returned error can't find the container with id 73812435990f1f46454dc2f07262def2dc4af98607890f65f960f6a01f27ae6f Jul 14 11:18:27.387824 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:27.387810 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 11:18:27.400623 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:27.400584 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal" event={"ID":"90fe358c1daa74a54db164d6ac35b548","Type":"ContainerStarted","Data":"73812435990f1f46454dc2f07262def2dc4af98607890f65f960f6a01f27ae6f"} Jul 14 11:18:27.401610 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:27.401587 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" event={"ID":"a8516bf7cc7b9f5f8ba46ac474d2b788","Type":"ContainerStarted","Data":"4db3ac0c1402f15cf9518ef4a5f29571a433073c4b3e34558fe5c2292ac2c949"} Jul 14 11:18:27.418761 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:27.418741 2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 11:18:27.445749 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:27.445729 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" Jul 14 11:18:27.467381 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:27.467359 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 11:18:27.469677 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:27.469663 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal" Jul 14 11:18:27.479347 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:27.479335 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 11:18:27.496177 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:27.496158 2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 11:18:27.538439 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:27.538385 2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 11:18:28.226808 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.226759 2571 apiserver.go:52] "Watching apiserver" Jul 14 11:18:28.237212 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.237183 2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 14 11:18:28.239138 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.239104 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-62zzs","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c","openshift-multus/multus-8m4rg","openshift-network-operator/iptables-alerter-w9vw6","openshift-ovn-kubernetes/ovnkube-node-5cglt","kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal","openshift-cluster-node-tuning-operator/tuned-zp5kb","openshift-dns/node-resolver-ttsdl","openshift-image-registry/node-ca-zmxv2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal","openshift-multus/multus-additional-cni-plugins-7784t","openshift-multus/network-metrics-daemon-fwpln","openshift-network-diagnostics/network-check-target-9hcgm","kube-system/global-pull-secret-syncer-882pd"] Jul 14 11:18:28.241136 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.240672 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.242068 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.242049 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:28.242214 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:28.242117 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hcgm" podUID="8d2c16f2-642c-4274-b5f2-3ee017c8b682" Jul 14 11:18:28.243588 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.243565 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 14 11:18:28.243828 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.243798 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dxzcl\"" Jul 14 11:18:28.243828 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.243820 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 14 11:18:28.245381 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.245355 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.246032 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.246014 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w9vw6" Jul 14 11:18:28.246122 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.246035 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 14 11:18:28.248047 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.247823 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fst6j\"" Jul 14 11:18:28.248047 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.247924 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 14 11:18:28.249598 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.248402 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 14 11:18:28.249598 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.248509 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 14 11:18:28.249598 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.248621 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 14 11:18:28.249598 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.249446 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 14 11:18:28.249598 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.249471 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 14 11:18:28.249896 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.249675 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-86q65\"" Jul 14 11:18:28.250599 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.250581 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 14 11:18:28.250989 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.250926 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.252598 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.252578 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:28.252708 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:28.252663 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-882pd" podUID="0fc0a645-d7c2-459a-8ecb-b0bae335b9ca" Jul 14 11:18:28.253727 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.253707 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zmxv2" Jul 14 11:18:28.254361 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.254341 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 14 11:18:28.254489 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.254466 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 14 11:18:28.254933 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.254915 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 14 11:18:28.255015 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.254933 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 14 11:18:28.255015 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.254990 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 14 11:18:28.255114 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.255068 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 14 11:18:28.255114 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.254933 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-st6tx\"" Jul 14 11:18:28.255287 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.255271 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-62zzs" Jul 14 11:18:28.255607 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.255586 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5be9ddca-55fc-4bc9-8a7a-aa245596c785-iptables-alerter-script\") pod \"iptables-alerter-w9vw6\" (UID: \"5be9ddca-55fc-4bc9-8a7a-aa245596c785\") " pod="openshift-network-operator/iptables-alerter-w9vw6" Jul 14 11:18:28.255717 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.255623 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5be9ddca-55fc-4bc9-8a7a-aa245596c785-host-slash\") pod \"iptables-alerter-w9vw6\" (UID: \"5be9ddca-55fc-4bc9-8a7a-aa245596c785\") " pod="openshift-network-operator/iptables-alerter-w9vw6" Jul 14 11:18:28.255717 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.255664 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5baf7735-3ad0-40de-b92f-1b77fc61c592-ovn-node-metrics-cert\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.255717 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.255690 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-os-release\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.255900 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.255728 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e67c4044-1964-428c-8a91-82e48aec2b66-cni-binary-copy\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.255900 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.255758 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-host-cni-bin\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.255900 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.255785 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-etc-sysconfig\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.255900 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.255805 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-etc-systemd\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.255900 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.255823 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.255900 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.255847 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5baf7735-3ad0-40de-b92f-1b77fc61c592-ovnkube-script-lib\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.255900 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.255871 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6b2h\" (UniqueName: \"kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h\") pod \"network-check-target-9hcgm\" (UID: \"8d2c16f2-642c-4274-b5f2-3ee017c8b682\") " pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:28.255900 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.255899 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-host-run-k8s-cni-cncf-io\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.256253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.255921 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5baf7735-3ad0-40de-b92f-1b77fc61c592-env-overrides\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.256253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.255943 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-etc-kubernetes\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.256253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.255965 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-etc-sysctl-conf\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.256253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.255986 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11a2bd40-f5eb-4981-8e86-4ec6014436ea-etc-tuned\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.256253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256008 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng8mk\" (UniqueName: \"kubernetes.io/projected/5be9ddca-55fc-4bc9-8a7a-aa245596c785-kube-api-access-ng8mk\") pod \"iptables-alerter-w9vw6\" (UID: \"5be9ddca-55fc-4bc9-8a7a-aa245596c785\") " pod="openshift-network-operator/iptables-alerter-w9vw6" Jul 14 11:18:28.256253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256032 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-run-systemd\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.256253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256055 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-run\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.256253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256078 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-host-var-lib-cni-multus\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.256253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256093 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-systemd-units\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.256253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256134 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-host-cni-netd\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.256253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256157 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11a2bd40-f5eb-4981-8e86-4ec6014436ea-tmp\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.256253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256180 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd2ht\" (UniqueName: \"kubernetes.io/projected/11a2bd40-f5eb-4981-8e86-4ec6014436ea-kube-api-access-dd2ht\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.256253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256204 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-lib-modules\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.256253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256239 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-cnibin\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.256253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256257 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-etc-kubernetes\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.256995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256275 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-host-run-netns\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.256995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256298 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-etc-modprobe-d\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.256995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256339 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 14 11:18:28.256995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256349 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-etc-sysctl-d\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.256995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256386 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-host-var-lib-kubelet\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.256995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256410 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-etc-openvswitch\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.256995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256427 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 14 11:18:28.256995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256431 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-run-ovn\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.256995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256455 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6fnd\" (UniqueName: \"kubernetes.io/projected/5baf7735-3ad0-40de-b92f-1b77fc61c592-kube-api-access-h6fnd\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.256995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256459 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 14 11:18:28.256995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256482 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-host\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.256995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256508 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-multus-socket-dir-parent\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.256995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256535 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-log-socket\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.256995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256561 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zqx8c\"" Jul 14 11:18:28.256995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256558 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-host-run-netns\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.256995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256607 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-host-var-lib-cni-bin\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.256995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256650 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-host-kubelet\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.256995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256678 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-var-lib-openvswitch\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.256995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256702 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-var-lib-kubelet\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.257887 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256726 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-multus-cni-dir\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.257887 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256777 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-hostroot\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.257887 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256802 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e67c4044-1964-428c-8a91-82e48aec2b66-multus-daemon-config\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.257887 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256826 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-host-run-multus-certs\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.257887 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256849 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-run-openvswitch\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.257887 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256872 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-node-log\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.257887 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256896 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnz6w\" (UniqueName: \"kubernetes.io/projected/e67c4044-1964-428c-8a91-82e48aec2b66-kube-api-access-lnz6w\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.257887 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256919 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5baf7735-3ad0-40de-b92f-1b77fc61c592-ovnkube-config\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.257887 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.256951 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-sys\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.257887 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.257000 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-multus-conf-dir\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.257887 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.257046 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-host-slash\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.257887 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.257088 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-host-run-ovn-kubernetes\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.257887 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.257111 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-system-cni-dir\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.257887 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.257302 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.258496 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.258223 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2btcc\"" Jul 14 11:18:28.258496 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.258274 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 14 11:18:28.258496 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.258274 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 14 11:18:28.258592 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.258530 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 11:18:28.258841 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.258794 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ttsdl" Jul 14 11:18:28.260007 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.259919 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gslqb\"" Jul 14 11:18:28.260007 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.259977 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 14 11:18:28.260165 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.260130 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 14 11:18:28.260225 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.259981 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 14 11:18:28.260536 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.260517 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.261666 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.261649 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 14 11:18:28.261826 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.261812 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 14 11:18:28.261902 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.261875 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dhqpf\"" Jul 14 11:18:28.261956 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.261944 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:28.262025 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:28.262002 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwpln" podUID="69592465-7d48-4eda-9662-361f7386ea86" Jul 14 11:18:28.262995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.262916 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 14 11:18:28.263238 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.263184 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 14 11:18:28.263238 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.263186 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qv2md\"" Jul 14 11:18:28.277883 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.277862 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kcgvv" Jul 14 11:18:28.290643 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.290611 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kcgvv" Jul 14 11:18:28.347292 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.347260 2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 14 11:18:28.357840 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.357807 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-lib-modules\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.357840 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.357841 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-cnibin\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.358033 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.357860 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-etc-kubernetes\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.358033 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.357883 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-host-run-netns\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.358033 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.357930 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/13f756c6-5b03-4308-8798-b0c75cc39c3b-sys-fs\") pod \"aws-ebs-csi-driver-node-9gg9c\" (UID: \"13f756c6-5b03-4308-8798-b0c75cc39c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.358033 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.357951 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-etc-kubernetes\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.358033 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.357974 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/13f756c6-5b03-4308-8798-b0c75cc39c3b-etc-selinux\") pod \"aws-ebs-csi-driver-node-9gg9c\" (UID: \"13f756c6-5b03-4308-8798-b0c75cc39c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.358033 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.357956 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-cnibin\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.358033 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.357997 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b7040e83-67d0-46ef-a022-91faba1e83b2-os-release\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.358033 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358009 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-lib-modules\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.358033 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358011 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-host-run-netns\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.358466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358047 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-etc-modprobe-d\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.358466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358071 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-etc-sysctl-d\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.358466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358107 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-host-var-lib-kubelet\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.358466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358146 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-etc-modprobe-d\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.358466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358154 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-etc-openvswitch\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.358466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358177 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-run-ovn\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.358466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358201 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6fnd\" (UniqueName: \"kubernetes.io/projected/5baf7735-3ad0-40de-b92f-1b77fc61c592-kube-api-access-h6fnd\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.358466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358206 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-etc-openvswitch\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.358466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358207 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-host-var-lib-kubelet\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.358466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358207 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-etc-sysctl-d\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.358466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358229 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-host\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.358466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358259 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-run-ovn\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.358466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358272 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-multus-socket-dir-parent\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.358466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358282 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-host\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.358466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358294 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-log-socket\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.358466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358332 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-log-socket\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.358466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358333 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4-serviceca\") pod \"node-ca-zmxv2\" (UID: \"b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4\") " pod="openshift-image-registry/node-ca-zmxv2" Jul 14 11:18:28.358466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358363 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9wtr\" (UniqueName: \"kubernetes.io/projected/fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a-kube-api-access-m9wtr\") pod \"node-resolver-ttsdl\" (UID: \"fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a\") " pod="openshift-dns/node-resolver-ttsdl" Jul 14 11:18:28.359253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358380 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-multus-socket-dir-parent\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.359253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358393 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-host-run-netns\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.359253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358430 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-host-run-netns\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.359253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358453 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-host-var-lib-cni-bin\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.359253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358479 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-host-kubelet\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.359253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358515 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-host-var-lib-cni-bin\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.359253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358523 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-host-kubelet\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.359253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358559 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-var-lib-openvswitch\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.359253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358584 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7040e83-67d0-46ef-a022-91faba1e83b2-cni-binary-copy\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.359253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358604 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-var-lib-kubelet\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.359253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358642 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-multus-cni-dir\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.359253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358653 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-var-lib-openvswitch\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.359253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358681 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-hostroot\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.359253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358696 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-var-lib-kubelet\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.359253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358700 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e67c4044-1964-428c-8a91-82e48aec2b66-multus-daemon-config\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.359253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358771 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-multus-cni-dir\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.359253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358776 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-hostroot\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.359253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358815 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-host-run-multus-certs\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.360096 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358857 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-run-openvswitch\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.360096 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358883 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3a64b363-5cc2-41ff-8a37-3df9b572ea88-konnectivity-ca\") pod \"konnectivity-agent-62zzs\" (UID: \"3a64b363-5cc2-41ff-8a37-3df9b572ea88\") " pod="kube-system/konnectivity-agent-62zzs" Jul 14 11:18:28.360096 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358943 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-host-run-multus-certs\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.360096 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358988 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/13f756c6-5b03-4308-8798-b0c75cc39c3b-registration-dir\") pod \"aws-ebs-csi-driver-node-9gg9c\" (UID: \"13f756c6-5b03-4308-8798-b0c75cc39c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.360096 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.358991 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-run-openvswitch\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.360096 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359024 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-node-log\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.360096 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359072 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7040e83-67d0-46ef-a022-91faba1e83b2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.360096 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359099 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8q2v2\" (UniqueName: \"kubernetes.io/projected/b7040e83-67d0-46ef-a022-91faba1e83b2-kube-api-access-8q2v2\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.360096 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359139 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnz6w\" (UniqueName: \"kubernetes.io/projected/e67c4044-1964-428c-8a91-82e48aec2b66-kube-api-access-lnz6w\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.360096 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359142 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-node-log\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.360096 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359187 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5baf7735-3ad0-40de-b92f-1b77fc61c592-ovnkube-config\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.360096 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359260 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e67c4044-1964-428c-8a91-82e48aec2b66-multus-daemon-config\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.360096 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359264 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9mlb\" (UniqueName: \"kubernetes.io/projected/13f756c6-5b03-4308-8798-b0c75cc39c3b-kube-api-access-x9mlb\") pod \"aws-ebs-csi-driver-node-9gg9c\" (UID: \"13f756c6-5b03-4308-8798-b0c75cc39c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.360096 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359314 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a-hosts-file\") pod \"node-resolver-ttsdl\" (UID: \"fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a\") " pod="openshift-dns/node-resolver-ttsdl" Jul 14 11:18:28.360096 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359338 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a-tmp-dir\") pod \"node-resolver-ttsdl\" (UID: \"fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a\") " pod="openshift-dns/node-resolver-ttsdl" Jul 14 11:18:28.360096 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359360 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-sys\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.360096 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359385 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-multus-conf-dir\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.360845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359413 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-host-slash\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.360845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359424 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-sys\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.360845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359437 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-host-run-ovn-kubernetes\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.360845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359453 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-multus-conf-dir\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.360845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359473 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7040e83-67d0-46ef-a022-91faba1e83b2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.360845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359491 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-system-cni-dir\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.360845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359499 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-host-run-ovn-kubernetes\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.360845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359505 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5be9ddca-55fc-4bc9-8a7a-aa245596c785-iptables-alerter-script\") pod \"iptables-alerter-w9vw6\" (UID: \"5be9ddca-55fc-4bc9-8a7a-aa245596c785\") " pod="openshift-network-operator/iptables-alerter-w9vw6" Jul 14 11:18:28.360845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359474 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-host-slash\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.360845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359537 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5be9ddca-55fc-4bc9-8a7a-aa245596c785-host-slash\") pod \"iptables-alerter-w9vw6\" (UID: \"5be9ddca-55fc-4bc9-8a7a-aa245596c785\") " pod="openshift-network-operator/iptables-alerter-w9vw6" Jul 14 11:18:28.360845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359567 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5baf7735-3ad0-40de-b92f-1b77fc61c592-ovn-node-metrics-cert\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.360845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359605 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8m4f\" (UniqueName: \"kubernetes.io/projected/69592465-7d48-4eda-9662-361f7386ea86-kube-api-access-h8m4f\") pod \"network-metrics-daemon-fwpln\" (UID: \"69592465-7d48-4eda-9662-361f7386ea86\") " pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:28.360845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359660 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-os-release\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.360845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359664 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-system-cni-dir\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.360845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359681 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e67c4044-1964-428c-8a91-82e48aec2b66-cni-binary-copy\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.360845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.359809 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5be9ddca-55fc-4bc9-8a7a-aa245596c785-host-slash\") pod \"iptables-alerter-w9vw6\" (UID: \"5be9ddca-55fc-4bc9-8a7a-aa245596c785\") " pod="openshift-network-operator/iptables-alerter-w9vw6" Jul 14 11:18:28.360845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.360586 2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 14 11:18:28.360845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.360787 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5be9ddca-55fc-4bc9-8a7a-aa245596c785-iptables-alerter-script\") pod \"iptables-alerter-w9vw6\" (UID: \"5be9ddca-55fc-4bc9-8a7a-aa245596c785\") " pod="openshift-network-operator/iptables-alerter-w9vw6" Jul 14 11:18:28.361566 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.360795 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e67c4044-1964-428c-8a91-82e48aec2b66-cni-binary-copy\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.361566 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.360794 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-os-release\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.361566 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.360482 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5baf7735-3ad0-40de-b92f-1b77fc61c592-ovnkube-config\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.361566 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.361018 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-host-cni-bin\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.361566 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.361084 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-kubelet-config\") pod \"global-pull-secret-syncer-882pd\" (UID: \"0fc0a645-d7c2-459a-8ecb-b0bae335b9ca\") " pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:28.361566 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.361118 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3a64b363-5cc2-41ff-8a37-3df9b572ea88-agent-certs\") pod \"konnectivity-agent-62zzs\" (UID: \"3a64b363-5cc2-41ff-8a37-3df9b572ea88\") " pod="kube-system/konnectivity-agent-62zzs" Jul 14 11:18:28.361566 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.361184 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-etc-sysconfig\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.361566 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.361245 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-etc-systemd\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.361566 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.361442 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-etc-sysconfig\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.361566 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.361485 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.361566 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.361509 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-host-cni-bin\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.362057 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.361585 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.362057 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.361623 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5baf7735-3ad0-40de-b92f-1b77fc61c592-ovnkube-script-lib\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.362057 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.361682 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-etc-systemd\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.362057 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.361677 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/13f756c6-5b03-4308-8798-b0c75cc39c3b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9gg9c\" (UID: \"13f756c6-5b03-4308-8798-b0c75cc39c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.362057 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.361821 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7040e83-67d0-46ef-a022-91faba1e83b2-system-cni-dir\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.362057 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.361959 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs\") pod \"network-metrics-daemon-fwpln\" (UID: \"69592465-7d48-4eda-9662-361f7386ea86\") " pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:28.362316 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.362108 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6b2h\" (UniqueName: \"kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h\") pod \"network-check-target-9hcgm\" (UID: \"8d2c16f2-642c-4274-b5f2-3ee017c8b682\") " pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:28.362316 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.362149 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-host-run-k8s-cni-cncf-io\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.362420 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.362378 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-host-run-k8s-cni-cncf-io\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.362835 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.362812 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret\") pod \"global-pull-secret-syncer-882pd\" (UID: \"0fc0a645-d7c2-459a-8ecb-b0bae335b9ca\") " pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:28.362921 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.362855 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4-host\") pod \"node-ca-zmxv2\" (UID: \"b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4\") " pod="openshift-image-registry/node-ca-zmxv2" Jul 14 11:18:28.362921 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.362889 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5baf7735-3ad0-40de-b92f-1b77fc61c592-env-overrides\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.362921 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.362917 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-etc-kubernetes\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.363051 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.362940 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-etc-sysctl-conf\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.363051 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.362970 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11a2bd40-f5eb-4981-8e86-4ec6014436ea-etc-tuned\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.363051 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.363009 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-etc-kubernetes\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.363051 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.362998 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng8mk\" (UniqueName: \"kubernetes.io/projected/5be9ddca-55fc-4bc9-8a7a-aa245596c785-kube-api-access-ng8mk\") pod \"iptables-alerter-w9vw6\" (UID: \"5be9ddca-55fc-4bc9-8a7a-aa245596c785\") " pod="openshift-network-operator/iptables-alerter-w9vw6" Jul 14 11:18:28.363227 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.363074 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-run-systemd\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.363227 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.363103 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-run\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.363227 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.363134 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-host-var-lib-cni-multus\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.363227 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.363163 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-systemd-units\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.363227 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.363168 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-etc-sysctl-conf\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.363227 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.362916 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5baf7735-3ad0-40de-b92f-1b77fc61c592-ovnkube-script-lib\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.363227 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.363189 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-host-cni-netd\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.363523 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.363233 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-dbus\") pod \"global-pull-secret-syncer-882pd\" (UID: \"0fc0a645-d7c2-459a-8ecb-b0bae335b9ca\") " pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:28.363523 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.363248 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-run-systemd\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.363523 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.363266 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdgcq\" (UniqueName: \"kubernetes.io/projected/b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4-kube-api-access-mdgcq\") pod \"node-ca-zmxv2\" (UID: \"b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4\") " pod="openshift-image-registry/node-ca-zmxv2" Jul 14 11:18:28.363523 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.363292 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/13f756c6-5b03-4308-8798-b0c75cc39c3b-device-dir\") pod \"aws-ebs-csi-driver-node-9gg9c\" (UID: \"13f756c6-5b03-4308-8798-b0c75cc39c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.363523 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.363308 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11a2bd40-f5eb-4981-8e86-4ec6014436ea-run\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.363523 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.363323 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7040e83-67d0-46ef-a022-91faba1e83b2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.363523 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.363355 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11a2bd40-f5eb-4981-8e86-4ec6014436ea-tmp\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.363523 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.363358 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e67c4044-1964-428c-8a91-82e48aec2b66-host-var-lib-cni-multus\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.363523 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.363383 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd2ht\" (UniqueName: \"kubernetes.io/projected/11a2bd40-f5eb-4981-8e86-4ec6014436ea-kube-api-access-dd2ht\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.363523 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.363402 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-systemd-units\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.363523 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.363408 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/13f756c6-5b03-4308-8798-b0c75cc39c3b-socket-dir\") pod \"aws-ebs-csi-driver-node-9gg9c\" (UID: \"13f756c6-5b03-4308-8798-b0c75cc39c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.363523 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.363444 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7040e83-67d0-46ef-a022-91faba1e83b2-cnibin\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.364079 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.363532 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5baf7735-3ad0-40de-b92f-1b77fc61c592-host-cni-netd\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.364126 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.364099 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5baf7735-3ad0-40de-b92f-1b77fc61c592-env-overrides\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.366520 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.366458 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5baf7735-3ad0-40de-b92f-1b77fc61c592-ovn-node-metrics-cert\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.367204 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.367185 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11a2bd40-f5eb-4981-8e86-4ec6014436ea-etc-tuned\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.367694 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.367674 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11a2bd40-f5eb-4981-8e86-4ec6014436ea-tmp\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.373529 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.373503 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnz6w\" (UniqueName: \"kubernetes.io/projected/e67c4044-1964-428c-8a91-82e48aec2b66-kube-api-access-lnz6w\") pod \"multus-8m4rg\" (UID: \"e67c4044-1964-428c-8a91-82e48aec2b66\") " pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.374162 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.374141 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6fnd\" (UniqueName: \"kubernetes.io/projected/5baf7735-3ad0-40de-b92f-1b77fc61c592-kube-api-access-h6fnd\") pod \"ovnkube-node-5cglt\" (UID: \"5baf7735-3ad0-40de-b92f-1b77fc61c592\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.378320 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:28.378297 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 11:18:28.378320 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:28.378321 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 11:18:28.378469 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:28.378335 2571 projected.go:196] Error preparing data for projected volume kube-api-access-n6b2h for pod openshift-network-diagnostics/network-check-target-9hcgm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:18:28.378469 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:28.378434 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h podName:8d2c16f2-642c-4274-b5f2-3ee017c8b682 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:28.878412963 +0000 UTC m=+3.039489422 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-n6b2h" (UniqueName: "kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h") pod "network-check-target-9hcgm" (UID: "8d2c16f2-642c-4274-b5f2-3ee017c8b682") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:18:28.380889 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.380861 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng8mk\" (UniqueName: \"kubernetes.io/projected/5be9ddca-55fc-4bc9-8a7a-aa245596c785-kube-api-access-ng8mk\") pod \"iptables-alerter-w9vw6\" (UID: \"5be9ddca-55fc-4bc9-8a7a-aa245596c785\") " pod="openshift-network-operator/iptables-alerter-w9vw6" Jul 14 11:18:28.382661 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.382642 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd2ht\" (UniqueName: \"kubernetes.io/projected/11a2bd40-f5eb-4981-8e86-4ec6014436ea-kube-api-access-dd2ht\") pod \"tuned-zp5kb\" (UID: \"11a2bd40-f5eb-4981-8e86-4ec6014436ea\") " pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.463822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.463796 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3a64b363-5cc2-41ff-8a37-3df9b572ea88-konnectivity-ca\") pod \"konnectivity-agent-62zzs\" (UID: \"3a64b363-5cc2-41ff-8a37-3df9b572ea88\") " pod="kube-system/konnectivity-agent-62zzs" Jul 14 11:18:28.463822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.463825 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/13f756c6-5b03-4308-8798-b0c75cc39c3b-registration-dir\") pod \"aws-ebs-csi-driver-node-9gg9c\" (UID: \"13f756c6-5b03-4308-8798-b0c75cc39c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.464073 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.463846 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7040e83-67d0-46ef-a022-91faba1e83b2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.464073 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.463872 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8q2v2\" (UniqueName: \"kubernetes.io/projected/b7040e83-67d0-46ef-a022-91faba1e83b2-kube-api-access-8q2v2\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.464073 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.463901 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9mlb\" (UniqueName: \"kubernetes.io/projected/13f756c6-5b03-4308-8798-b0c75cc39c3b-kube-api-access-x9mlb\") pod \"aws-ebs-csi-driver-node-9gg9c\" (UID: \"13f756c6-5b03-4308-8798-b0c75cc39c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.464073 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.463924 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a-hosts-file\") pod \"node-resolver-ttsdl\" (UID: \"fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a\") " pod="openshift-dns/node-resolver-ttsdl" Jul 14 11:18:28.464073 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.463985 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a-hosts-file\") pod \"node-resolver-ttsdl\" (UID: \"fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a\") " pod="openshift-dns/node-resolver-ttsdl" Jul 14 11:18:28.464073 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464001 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/13f756c6-5b03-4308-8798-b0c75cc39c3b-registration-dir\") pod \"aws-ebs-csi-driver-node-9gg9c\" (UID: \"13f756c6-5b03-4308-8798-b0c75cc39c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.464327 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464088 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a-tmp-dir\") pod \"node-resolver-ttsdl\" (UID: \"fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a\") " pod="openshift-dns/node-resolver-ttsdl" Jul 14 11:18:28.464327 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464123 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7040e83-67d0-46ef-a022-91faba1e83b2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.464327 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464154 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8m4f\" (UniqueName: \"kubernetes.io/projected/69592465-7d48-4eda-9662-361f7386ea86-kube-api-access-h8m4f\") pod \"network-metrics-daemon-fwpln\" (UID: \"69592465-7d48-4eda-9662-361f7386ea86\") " pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:28.464327 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464183 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-kubelet-config\") pod \"global-pull-secret-syncer-882pd\" (UID: \"0fc0a645-d7c2-459a-8ecb-b0bae335b9ca\") " pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:28.464327 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464212 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3a64b363-5cc2-41ff-8a37-3df9b572ea88-agent-certs\") pod \"konnectivity-agent-62zzs\" (UID: \"3a64b363-5cc2-41ff-8a37-3df9b572ea88\") " pod="kube-system/konnectivity-agent-62zzs" Jul 14 11:18:28.464327 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464239 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/13f756c6-5b03-4308-8798-b0c75cc39c3b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9gg9c\" (UID: \"13f756c6-5b03-4308-8798-b0c75cc39c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.464327 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464279 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7040e83-67d0-46ef-a022-91faba1e83b2-system-cni-dir\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.464327 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464302 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs\") pod \"network-metrics-daemon-fwpln\" (UID: \"69592465-7d48-4eda-9662-361f7386ea86\") " pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:28.464603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464340 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret\") pod \"global-pull-secret-syncer-882pd\" (UID: \"0fc0a645-d7c2-459a-8ecb-b0bae335b9ca\") " pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:28.464603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464360 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4-host\") pod \"node-ca-zmxv2\" (UID: \"b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4\") " pod="openshift-image-registry/node-ca-zmxv2" Jul 14 11:18:28.464603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464380 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-dbus\") pod \"global-pull-secret-syncer-882pd\" (UID: \"0fc0a645-d7c2-459a-8ecb-b0bae335b9ca\") " pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:28.464603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464397 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdgcq\" (UniqueName: \"kubernetes.io/projected/b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4-kube-api-access-mdgcq\") pod \"node-ca-zmxv2\" (UID: \"b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4\") " pod="openshift-image-registry/node-ca-zmxv2" Jul 14 11:18:28.464603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464449 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/13f756c6-5b03-4308-8798-b0c75cc39c3b-device-dir\") pod \"aws-ebs-csi-driver-node-9gg9c\" (UID: \"13f756c6-5b03-4308-8798-b0c75cc39c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.464603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464448 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a-tmp-dir\") pod \"node-resolver-ttsdl\" (UID: \"fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a\") " pod="openshift-dns/node-resolver-ttsdl" Jul 14 11:18:28.464603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464465 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7040e83-67d0-46ef-a022-91faba1e83b2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.464603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464454 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3a64b363-5cc2-41ff-8a37-3df9b572ea88-konnectivity-ca\") pod \"konnectivity-agent-62zzs\" (UID: \"3a64b363-5cc2-41ff-8a37-3df9b572ea88\") " pod="kube-system/konnectivity-agent-62zzs" Jul 14 11:18:28.464603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464509 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/13f756c6-5b03-4308-8798-b0c75cc39c3b-socket-dir\") pod \"aws-ebs-csi-driver-node-9gg9c\" (UID: \"13f756c6-5b03-4308-8798-b0c75cc39c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.464603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464520 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-kubelet-config\") pod \"global-pull-secret-syncer-882pd\" (UID: \"0fc0a645-d7c2-459a-8ecb-b0bae335b9ca\") " pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:28.464603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464539 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7040e83-67d0-46ef-a022-91faba1e83b2-cnibin\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.464603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464565 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7040e83-67d0-46ef-a022-91faba1e83b2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.464603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464579 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7040e83-67d0-46ef-a022-91faba1e83b2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.464603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464568 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/13f756c6-5b03-4308-8798-b0c75cc39c3b-sys-fs\") pod \"aws-ebs-csi-driver-node-9gg9c\" (UID: \"13f756c6-5b03-4308-8798-b0c75cc39c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.465074 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464613 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/13f756c6-5b03-4308-8798-b0c75cc39c3b-sys-fs\") pod \"aws-ebs-csi-driver-node-9gg9c\" (UID: \"13f756c6-5b03-4308-8798-b0c75cc39c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.465074 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464613 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/13f756c6-5b03-4308-8798-b0c75cc39c3b-etc-selinux\") pod \"aws-ebs-csi-driver-node-9gg9c\" (UID: \"13f756c6-5b03-4308-8798-b0c75cc39c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.465074 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464667 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b7040e83-67d0-46ef-a022-91faba1e83b2-os-release\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.465074 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464715 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7040e83-67d0-46ef-a022-91faba1e83b2-cnibin\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.465074 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464725 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/13f756c6-5b03-4308-8798-b0c75cc39c3b-etc-selinux\") pod \"aws-ebs-csi-driver-node-9gg9c\" (UID: \"13f756c6-5b03-4308-8798-b0c75cc39c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.465074 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:28.464755 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 11:18:28.465074 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464787 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7040e83-67d0-46ef-a022-91faba1e83b2-system-cni-dir\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.465074 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464793 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/13f756c6-5b03-4308-8798-b0c75cc39c3b-socket-dir\") pod \"aws-ebs-csi-driver-node-9gg9c\" (UID: \"13f756c6-5b03-4308-8798-b0c75cc39c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.465074 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464817 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-dbus\") pod \"global-pull-secret-syncer-882pd\" (UID: \"0fc0a645-d7c2-459a-8ecb-b0bae335b9ca\") " pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:28.465074 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464830 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/13f756c6-5b03-4308-8798-b0c75cc39c3b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9gg9c\" (UID: \"13f756c6-5b03-4308-8798-b0c75cc39c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.465074 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:28.464833 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret podName:0fc0a645-d7c2-459a-8ecb-b0bae335b9ca nodeName:}" failed. No retries permitted until 2026-07-14 11:18:28.964815686 +0000 UTC m=+3.125892138 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret") pod "global-pull-secret-syncer-882pd" (UID: "0fc0a645-d7c2-459a-8ecb-b0bae335b9ca") : object "kube-system"/"original-pull-secret" not registered Jul 14 11:18:28.465074 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464861 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4-serviceca\") pod \"node-ca-zmxv2\" (UID: \"b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4\") " pod="openshift-image-registry/node-ca-zmxv2" Jul 14 11:18:28.465074 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464872 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b7040e83-67d0-46ef-a022-91faba1e83b2-os-release\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.465074 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464762 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4-host\") pod \"node-ca-zmxv2\" (UID: \"b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4\") " pod="openshift-image-registry/node-ca-zmxv2" Jul 14 11:18:28.465074 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464874 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/13f756c6-5b03-4308-8798-b0c75cc39c3b-device-dir\") pod \"aws-ebs-csi-driver-node-9gg9c\" (UID: \"13f756c6-5b03-4308-8798-b0c75cc39c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.465074 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464886 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9wtr\" (UniqueName: \"kubernetes.io/projected/fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a-kube-api-access-m9wtr\") pod \"node-resolver-ttsdl\" (UID: \"fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a\") " pod="openshift-dns/node-resolver-ttsdl" Jul 14 11:18:28.465074 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.464928 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7040e83-67d0-46ef-a022-91faba1e83b2-cni-binary-copy\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.465693 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:28.464945 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:18:28.465693 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:28.464996 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs podName:69592465-7d48-4eda-9662-361f7386ea86 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:28.96498207 +0000 UTC m=+3.126058530 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs") pod "network-metrics-daemon-fwpln" (UID: "69592465-7d48-4eda-9662-361f7386ea86") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:18:28.465693 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.465279 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4-serviceca\") pod \"node-ca-zmxv2\" (UID: \"b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4\") " pod="openshift-image-registry/node-ca-zmxv2" Jul 14 11:18:28.465693 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.465386 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7040e83-67d0-46ef-a022-91faba1e83b2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.465693 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.465477 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7040e83-67d0-46ef-a022-91faba1e83b2-cni-binary-copy\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.466922 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.466902 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3a64b363-5cc2-41ff-8a37-3df9b572ea88-agent-certs\") pod \"konnectivity-agent-62zzs\" (UID: \"3a64b363-5cc2-41ff-8a37-3df9b572ea88\") " pod="kube-system/konnectivity-agent-62zzs" Jul 14 11:18:28.479902 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.479843 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdgcq\" (UniqueName: \"kubernetes.io/projected/b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4-kube-api-access-mdgcq\") pod \"node-ca-zmxv2\" (UID: \"b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4\") " pod="openshift-image-registry/node-ca-zmxv2" Jul 14 11:18:28.480167 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.480144 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8q2v2\" (UniqueName: \"kubernetes.io/projected/b7040e83-67d0-46ef-a022-91faba1e83b2-kube-api-access-8q2v2\") pod \"multus-additional-cni-plugins-7784t\" (UID: \"b7040e83-67d0-46ef-a022-91faba1e83b2\") " pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.480324 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.480297 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8m4f\" (UniqueName: \"kubernetes.io/projected/69592465-7d48-4eda-9662-361f7386ea86-kube-api-access-h8m4f\") pod \"network-metrics-daemon-fwpln\" (UID: \"69592465-7d48-4eda-9662-361f7386ea86\") " pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:28.480904 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.480884 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9wtr\" (UniqueName: \"kubernetes.io/projected/fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a-kube-api-access-m9wtr\") pod \"node-resolver-ttsdl\" (UID: \"fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a\") " pod="openshift-dns/node-resolver-ttsdl" Jul 14 11:18:28.481001 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.480979 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9mlb\" (UniqueName: \"kubernetes.io/projected/13f756c6-5b03-4308-8798-b0c75cc39c3b-kube-api-access-x9mlb\") pod \"aws-ebs-csi-driver-node-9gg9c\" (UID: \"13f756c6-5b03-4308-8798-b0c75cc39c3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.555042 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.555013 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" Jul 14 11:18:28.564405 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.564380 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8m4rg" Jul 14 11:18:28.574271 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.574249 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-w9vw6" Jul 14 11:18:28.585183 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.585159 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:28.591936 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.591914 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zmxv2" Jul 14 11:18:28.597663 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.597643 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-62zzs" Jul 14 11:18:28.605371 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.605355 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" Jul 14 11:18:28.612133 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.612113 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ttsdl" Jul 14 11:18:28.616963 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.616940 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7784t" Jul 14 11:18:28.783427 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.783346 2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 11:18:28.968668 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.968622 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs\") pod \"network-metrics-daemon-fwpln\" (UID: \"69592465-7d48-4eda-9662-361f7386ea86\") " pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:28.968796 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.968681 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6b2h\" (UniqueName: \"kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h\") pod \"network-check-target-9hcgm\" (UID: \"8d2c16f2-642c-4274-b5f2-3ee017c8b682\") " pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:28.968796 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:28.968719 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret\") pod \"global-pull-secret-syncer-882pd\" (UID: \"0fc0a645-d7c2-459a-8ecb-b0bae335b9ca\") " pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:28.968796 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:28.968736 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:18:28.968796 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:28.968795 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs podName:69592465-7d48-4eda-9662-361f7386ea86 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:29.96877924 +0000 UTC m=+4.129855685 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs") pod "network-metrics-daemon-fwpln" (UID: "69592465-7d48-4eda-9662-361f7386ea86") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:18:28.968956 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:28.968867 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 11:18:28.968956 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:28.968872 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 11:18:28.968956 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:28.968893 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 11:18:28.968956 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:28.968905 2571 projected.go:196] Error preparing data for projected volume kube-api-access-n6b2h for pod openshift-network-diagnostics/network-check-target-9hcgm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:18:28.968956 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:28.968953 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret podName:0fc0a645-d7c2-459a-8ecb-b0bae335b9ca nodeName:}" failed. No retries permitted until 2026-07-14 11:18:29.96890632 +0000 UTC m=+4.129982783 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret") pod "global-pull-secret-syncer-882pd" (UID: "0fc0a645-d7c2-459a-8ecb-b0bae335b9ca") : object "kube-system"/"original-pull-secret" not registered Jul 14 11:18:28.969116 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:28.968968 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h podName:8d2c16f2-642c-4274-b5f2-3ee017c8b682 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:29.968960432 +0000 UTC m=+4.130036878 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-n6b2h" (UniqueName: "kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h") pod "network-check-target-9hcgm" (UID: "8d2c16f2-642c-4274-b5f2-3ee017c8b682") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:18:29.293414 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:29.293381 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 11:13:28 +0000 UTC" deadline="2028-01-31 00:56:15.752331288 +0000 UTC" Jul 14 11:18:29.293414 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:29.293412 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13573h37m46.458923056s" Jul 14 11:18:29.329987 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:29.329951 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode67c4044_1964_428c_8a91_82e48aec2b66.slice/crio-e9fbd53f3778eb85eea8960964d6cec39b7fd43f235bcea13620658a3d8b0b68 WatchSource:0}: Error finding container e9fbd53f3778eb85eea8960964d6cec39b7fd43f235bcea13620658a3d8b0b68: Status 404 returned error can't find the container with id e9fbd53f3778eb85eea8960964d6cec39b7fd43f235bcea13620658a3d8b0b68 Jul 14 11:18:29.330690 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:29.330667 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5baf7735_3ad0_40de_b92f_1b77fc61c592.slice/crio-97f233e2b2ac539b63e69830db1dcb60922f6a81e3f5bfc6f4fae732e9cf9aa7 WatchSource:0}: Error finding container 97f233e2b2ac539b63e69830db1dcb60922f6a81e3f5bfc6f4fae732e9cf9aa7: Status 404 returned error can't find the container with id 97f233e2b2ac539b63e69830db1dcb60922f6a81e3f5bfc6f4fae732e9cf9aa7 Jul 14 11:18:29.331533 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:29.331485 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7040e83_67d0_46ef_a022_91faba1e83b2.slice/crio-3e34246048699c90e03e70c98d68ae7c5eeace28b22e175dc6d7bb8482ad2c02 WatchSource:0}: Error finding container 3e34246048699c90e03e70c98d68ae7c5eeace28b22e175dc6d7bb8482ad2c02: Status 404 returned error can't find the container with id 3e34246048699c90e03e70c98d68ae7c5eeace28b22e175dc6d7bb8482ad2c02 Jul 14 11:18:29.335596 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:29.332773 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5be9ddca_55fc_4bc9_8a7a_aa245596c785.slice/crio-2081c15f07f592e24b1c2bdbbbdc0e2128e2cb42287bb0621018d2e422ea362f WatchSource:0}: Error finding container 2081c15f07f592e24b1c2bdbbbdc0e2128e2cb42287bb0621018d2e422ea362f: Status 404 returned error can't find the container with id 2081c15f07f592e24b1c2bdbbbdc0e2128e2cb42287bb0621018d2e422ea362f Jul 14 11:18:29.338486 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:29.338456 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb27010c7_44aa_40f5_8aa3_ae1c20e6c7f4.slice/crio-bc6325f1b230c1456418a3fc4dcb187b3ff9bc1abb8111a460d99a60056c2849 WatchSource:0}: Error finding container bc6325f1b230c1456418a3fc4dcb187b3ff9bc1abb8111a460d99a60056c2849: Status 404 returned error can't find the container with id bc6325f1b230c1456418a3fc4dcb187b3ff9bc1abb8111a460d99a60056c2849 Jul 14 11:18:29.339618 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:29.339311 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a64b363_5cc2_41ff_8a37_3df9b572ea88.slice/crio-4ee0acaf8f0592f648cb316d1fdf79a85bef32268bf0ab449667c1dd139be226 WatchSource:0}: Error finding container 4ee0acaf8f0592f648cb316d1fdf79a85bef32268bf0ab449667c1dd139be226: Status 404 returned error can't find the container with id 4ee0acaf8f0592f648cb316d1fdf79a85bef32268bf0ab449667c1dd139be226 Jul 14 11:18:29.340583 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:18:29.340562 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa67ad6b_3e7d_4fb4_9f0e_c186cfa8912a.slice/crio-07255f6ad3c9922e2085e475e726768499d4fd2f978426e6c7686c445079f72f WatchSource:0}: Error finding container 07255f6ad3c9922e2085e475e726768499d4fd2f978426e6c7686c445079f72f: Status 404 returned error can't find the container with id 07255f6ad3c9922e2085e475e726768499d4fd2f978426e6c7686c445079f72f Jul 14 11:18:29.397853 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:29.397720 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:29.397949 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:29.397930 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-882pd" podUID="0fc0a645-d7c2-459a-8ecb-b0bae335b9ca" Jul 14 11:18:29.405376 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:29.405350 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zmxv2" event={"ID":"b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4","Type":"ContainerStarted","Data":"bc6325f1b230c1456418a3fc4dcb187b3ff9bc1abb8111a460d99a60056c2849"} Jul 14 11:18:29.406470 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:29.406448 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w9vw6" event={"ID":"5be9ddca-55fc-4bc9-8a7a-aa245596c785","Type":"ContainerStarted","Data":"2081c15f07f592e24b1c2bdbbbdc0e2128e2cb42287bb0621018d2e422ea362f"} Jul 14 11:18:29.407524 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:29.407503 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7784t" event={"ID":"b7040e83-67d0-46ef-a022-91faba1e83b2","Type":"ContainerStarted","Data":"3e34246048699c90e03e70c98d68ae7c5eeace28b22e175dc6d7bb8482ad2c02"} Jul 14 11:18:29.408684 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:29.408621 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" event={"ID":"11a2bd40-f5eb-4981-8e86-4ec6014436ea","Type":"ContainerStarted","Data":"e731ae0af6597bc2d6c680d89013efa98f47966b6c9061d5f4710e6bb5f8eaba"} Jul 14 11:18:29.409677 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:29.409655 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-62zzs" event={"ID":"3a64b363-5cc2-41ff-8a37-3df9b572ea88","Type":"ContainerStarted","Data":"4ee0acaf8f0592f648cb316d1fdf79a85bef32268bf0ab449667c1dd139be226"} Jul 14 11:18:29.410744 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:29.410719 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" event={"ID":"13f756c6-5b03-4308-8798-b0c75cc39c3b","Type":"ContainerStarted","Data":"21364f31c516aaa0a002916f43f9d034b5d2c37caa89ed7efae1661816a87226"} Jul 14 11:18:29.411790 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:29.411770 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" event={"ID":"5baf7735-3ad0-40de-b92f-1b77fc61c592","Type":"ContainerStarted","Data":"97f233e2b2ac539b63e69830db1dcb60922f6a81e3f5bfc6f4fae732e9cf9aa7"} Jul 14 11:18:29.412757 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:29.412733 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8m4rg" event={"ID":"e67c4044-1964-428c-8a91-82e48aec2b66","Type":"ContainerStarted","Data":"e9fbd53f3778eb85eea8960964d6cec39b7fd43f235bcea13620658a3d8b0b68"} Jul 14 11:18:29.414010 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:29.413991 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ttsdl" event={"ID":"fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a","Type":"ContainerStarted","Data":"07255f6ad3c9922e2085e475e726768499d4fd2f978426e6c7686c445079f72f"} Jul 14 11:18:29.977468 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:29.977378 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs\") pod \"network-metrics-daemon-fwpln\" (UID: \"69592465-7d48-4eda-9662-361f7386ea86\") " pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:29.977468 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:29.977428 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6b2h\" (UniqueName: \"kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h\") pod \"network-check-target-9hcgm\" (UID: \"8d2c16f2-642c-4274-b5f2-3ee017c8b682\") " pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:29.977468 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:29.977456 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret\") pod \"global-pull-secret-syncer-882pd\" (UID: \"0fc0a645-d7c2-459a-8ecb-b0bae335b9ca\") " pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:29.977781 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:29.977581 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 11:18:29.977781 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:29.977655 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret podName:0fc0a645-d7c2-459a-8ecb-b0bae335b9ca nodeName:}" failed. No retries permitted until 2026-07-14 11:18:31.97761967 +0000 UTC m=+6.138696119 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret") pod "global-pull-secret-syncer-882pd" (UID: "0fc0a645-d7c2-459a-8ecb-b0bae335b9ca") : object "kube-system"/"original-pull-secret" not registered Jul 14 11:18:29.978050 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:29.978029 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:18:29.978119 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:29.978088 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs podName:69592465-7d48-4eda-9662-361f7386ea86 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:31.978071218 +0000 UTC m=+6.139147663 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs") pod "network-metrics-daemon-fwpln" (UID: "69592465-7d48-4eda-9662-361f7386ea86") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:18:29.978197 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:29.978173 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 11:18:29.978197 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:29.978190 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 11:18:29.978285 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:29.978202 2571 projected.go:196] Error preparing data for projected volume kube-api-access-n6b2h for pod openshift-network-diagnostics/network-check-target-9hcgm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:18:29.978285 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:29.978236 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h podName:8d2c16f2-642c-4274-b5f2-3ee017c8b682 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:31.978225433 +0000 UTC m=+6.139301881 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-n6b2h" (UniqueName: "kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h") pod "network-check-target-9hcgm" (UID: "8d2c16f2-642c-4274-b5f2-3ee017c8b682") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:18:30.294225 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:30.294103 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 11:13:28 +0000 UTC" deadline="2028-04-04 09:17:58.128257198 +0000 UTC" Jul 14 11:18:30.294225 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:30.294158 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15117h59m27.834102882s" Jul 14 11:18:30.397398 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:30.397364 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:30.397567 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:30.397511 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwpln" podUID="69592465-7d48-4eda-9662-361f7386ea86" Jul 14 11:18:30.400083 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:30.400051 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:30.400203 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:30.400170 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hcgm" podUID="8d2c16f2-642c-4274-b5f2-3ee017c8b682" Jul 14 11:18:30.427212 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:30.427178 2571 generic.go:356] "Generic (PLEG): container finished" podID="a8516bf7cc7b9f5f8ba46ac474d2b788" containerID="a18c8d7379ca30c4488fdb43bfbec5690c03d9439ac9eeb253ecb6851ea240c3" exitCode=0 Jul 14 11:18:30.427360 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:30.427270 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" event={"ID":"a8516bf7cc7b9f5f8ba46ac474d2b788","Type":"ContainerDied","Data":"a18c8d7379ca30c4488fdb43bfbec5690c03d9439ac9eeb253ecb6851ea240c3"} Jul 14 11:18:30.446672 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:30.445949 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal" event={"ID":"90fe358c1daa74a54db164d6ac35b548","Type":"ContainerStarted","Data":"31cd5a90e9142280d630d60ab7d4de3364389c66bdefbae3c8c5b6ee4557c28f"} Jul 14 11:18:31.398194 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:31.397950 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:31.398194 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:31.398079 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-882pd" podUID="0fc0a645-d7c2-459a-8ecb-b0bae335b9ca" Jul 14 11:18:31.466793 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:31.466687 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" event={"ID":"a8516bf7cc7b9f5f8ba46ac474d2b788","Type":"ContainerStarted","Data":"d09413a4e85d6b343f0e5c42ae21735864c83f8a7a43992112566af9ea77545a"} Jul 14 11:18:31.486405 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:31.486346 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal" podStartSLOduration=4.486327028 podStartE2EDuration="4.486327028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:30.462369325 +0000 UTC m=+4.623445793" watchObservedRunningTime="2026-07-14 11:18:31.486327028 +0000 UTC m=+5.647403495" Jul 14 11:18:31.997854 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:31.997814 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs\") pod \"network-metrics-daemon-fwpln\" (UID: \"69592465-7d48-4eda-9662-361f7386ea86\") " pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:31.998038 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:31.997869 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6b2h\" (UniqueName: \"kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h\") pod \"network-check-target-9hcgm\" (UID: \"8d2c16f2-642c-4274-b5f2-3ee017c8b682\") " pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:31.998038 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:31.997900 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret\") pod \"global-pull-secret-syncer-882pd\" (UID: \"0fc0a645-d7c2-459a-8ecb-b0bae335b9ca\") " pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:31.998038 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:31.998021 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 11:18:31.998217 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:31.998083 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret podName:0fc0a645-d7c2-459a-8ecb-b0bae335b9ca nodeName:}" failed. No retries permitted until 2026-07-14 11:18:35.998064177 +0000 UTC m=+10.159140636 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret") pod "global-pull-secret-syncer-882pd" (UID: "0fc0a645-d7c2-459a-8ecb-b0bae335b9ca") : object "kube-system"/"original-pull-secret" not registered Jul 14 11:18:31.998505 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:31.998484 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:18:31.998606 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:31.998539 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs podName:69592465-7d48-4eda-9662-361f7386ea86 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:35.998523257 +0000 UTC m=+10.159599716 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs") pod "network-metrics-daemon-fwpln" (UID: "69592465-7d48-4eda-9662-361f7386ea86") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:18:31.998726 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:31.998618 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 11:18:31.998726 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:31.998651 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 11:18:31.998726 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:31.998663 2571 projected.go:196] Error preparing data for projected volume kube-api-access-n6b2h for pod openshift-network-diagnostics/network-check-target-9hcgm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:18:31.998726 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:31.998697 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h podName:8d2c16f2-642c-4274-b5f2-3ee017c8b682 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:35.998686588 +0000 UTC m=+10.159763049 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-n6b2h" (UniqueName: "kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h") pod "network-check-target-9hcgm" (UID: "8d2c16f2-642c-4274-b5f2-3ee017c8b682") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:18:32.401320 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:32.400571 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:32.401320 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:32.400731 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwpln" podUID="69592465-7d48-4eda-9662-361f7386ea86" Jul 14 11:18:32.401320 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:32.401113 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:32.401320 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:32.401202 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hcgm" podUID="8d2c16f2-642c-4274-b5f2-3ee017c8b682" Jul 14 11:18:33.397794 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:33.397762 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:33.397962 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:33.397926 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-882pd" podUID="0fc0a645-d7c2-459a-8ecb-b0bae335b9ca" Jul 14 11:18:34.397892 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:34.397851 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:34.398341 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:34.398009 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwpln" podUID="69592465-7d48-4eda-9662-361f7386ea86" Jul 14 11:18:34.398405 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:34.398339 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:34.398459 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:34.398438 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hcgm" podUID="8d2c16f2-642c-4274-b5f2-3ee017c8b682" Jul 14 11:18:35.398172 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:35.397660 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:35.398172 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:35.397794 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-882pd" podUID="0fc0a645-d7c2-459a-8ecb-b0bae335b9ca" Jul 14 11:18:36.032388 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:36.032252 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs\") pod \"network-metrics-daemon-fwpln\" (UID: \"69592465-7d48-4eda-9662-361f7386ea86\") " pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:36.032388 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:36.032302 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6b2h\" (UniqueName: \"kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h\") pod \"network-check-target-9hcgm\" (UID: \"8d2c16f2-642c-4274-b5f2-3ee017c8b682\") " pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:36.032388 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:36.032330 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret\") pod \"global-pull-secret-syncer-882pd\" (UID: \"0fc0a645-d7c2-459a-8ecb-b0bae335b9ca\") " pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:36.032711 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:36.032443 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:18:36.032711 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:36.032481 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 11:18:36.032711 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:36.032502 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs podName:69592465-7d48-4eda-9662-361f7386ea86 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:44.032485198 +0000 UTC m=+18.193561661 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs") pod "network-metrics-daemon-fwpln" (UID: "69592465-7d48-4eda-9662-361f7386ea86") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:18:36.032711 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:36.032501 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 11:18:36.032711 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:36.032566 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret podName:0fc0a645-d7c2-459a-8ecb-b0bae335b9ca nodeName:}" failed. No retries permitted until 2026-07-14 11:18:44.032549089 +0000 UTC m=+18.193625541 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret") pod "global-pull-secret-syncer-882pd" (UID: "0fc0a645-d7c2-459a-8ecb-b0bae335b9ca") : object "kube-system"/"original-pull-secret" not registered Jul 14 11:18:36.032711 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:36.032508 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 11:18:36.032711 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:36.032608 2571 projected.go:196] Error preparing data for projected volume kube-api-access-n6b2h for pod openshift-network-diagnostics/network-check-target-9hcgm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:18:36.032711 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:36.032660 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h podName:8d2c16f2-642c-4274-b5f2-3ee017c8b682 nodeName:}" failed. No retries permitted until 2026-07-14 11:18:44.032647702 +0000 UTC m=+18.193724150 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-n6b2h" (UniqueName: "kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h") pod "network-check-target-9hcgm" (UID: "8d2c16f2-642c-4274-b5f2-3ee017c8b682") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:18:36.399334 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:36.398595 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:36.399334 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:36.398724 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hcgm" podUID="8d2c16f2-642c-4274-b5f2-3ee017c8b682" Jul 14 11:18:36.399334 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:36.399133 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:36.399334 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:36.399237 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwpln" podUID="69592465-7d48-4eda-9662-361f7386ea86" Jul 14 11:18:37.397797 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:37.397768 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:37.397954 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:37.397890 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-882pd" podUID="0fc0a645-d7c2-459a-8ecb-b0bae335b9ca" Jul 14 11:18:38.398063 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:38.398027 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:38.398541 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:38.398034 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:38.398541 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:38.398185 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwpln" podUID="69592465-7d48-4eda-9662-361f7386ea86" Jul 14 11:18:38.398541 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:38.398261 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hcgm" podUID="8d2c16f2-642c-4274-b5f2-3ee017c8b682" Jul 14 11:18:39.398025 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:39.397989 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:39.398201 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:39.398124 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-882pd" podUID="0fc0a645-d7c2-459a-8ecb-b0bae335b9ca" Jul 14 11:18:40.400845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:40.400799 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:40.400845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:40.400843 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:40.401329 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:40.400937 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hcgm" podUID="8d2c16f2-642c-4274-b5f2-3ee017c8b682" Jul 14 11:18:40.401329 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:40.401094 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwpln" podUID="69592465-7d48-4eda-9662-361f7386ea86" Jul 14 11:18:41.397607 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:41.397572 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:41.397782 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:41.397708 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-882pd" podUID="0fc0a645-d7c2-459a-8ecb-b0bae335b9ca" Jul 14 11:18:42.398243 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:42.398209 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:42.398690 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:42.398262 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:42.398690 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:42.398363 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwpln" podUID="69592465-7d48-4eda-9662-361f7386ea86" Jul 14 11:18:42.398690 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:42.398487 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hcgm" podUID="8d2c16f2-642c-4274-b5f2-3ee017c8b682" Jul 14 11:18:43.397777 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:43.397739 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:43.397973 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:43.397879 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-882pd" podUID="0fc0a645-d7c2-459a-8ecb-b0bae335b9ca" Jul 14 11:18:44.096945 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:44.096903 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs\") pod \"network-metrics-daemon-fwpln\" (UID: \"69592465-7d48-4eda-9662-361f7386ea86\") " pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:44.096945 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:44.096950 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6b2h\" (UniqueName: \"kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h\") pod \"network-check-target-9hcgm\" (UID: \"8d2c16f2-642c-4274-b5f2-3ee017c8b682\") " pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:44.097446 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:44.097056 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret\") pod \"global-pull-secret-syncer-882pd\" (UID: \"0fc0a645-d7c2-459a-8ecb-b0bae335b9ca\") " pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:44.097446 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:44.097097 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:18:44.097446 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:44.097155 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 11:18:44.097446 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:44.097186 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs podName:69592465-7d48-4eda-9662-361f7386ea86 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:00.097163931 +0000 UTC m=+34.258240376 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs") pod "network-metrics-daemon-fwpln" (UID: "69592465-7d48-4eda-9662-361f7386ea86") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:18:44.097446 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:44.097188 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 11:18:44.097446 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:44.097208 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret podName:0fc0a645-d7c2-459a-8ecb-b0bae335b9ca nodeName:}" failed. No retries permitted until 2026-07-14 11:19:00.097196689 +0000 UTC m=+34.258273138 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret") pod "global-pull-secret-syncer-882pd" (UID: "0fc0a645-d7c2-459a-8ecb-b0bae335b9ca") : object "kube-system"/"original-pull-secret" not registered Jul 14 11:18:44.097446 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:44.097214 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 11:18:44.097446 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:44.097230 2571 projected.go:196] Error preparing data for projected volume kube-api-access-n6b2h for pod openshift-network-diagnostics/network-check-target-9hcgm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:18:44.097446 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:44.097283 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h podName:8d2c16f2-642c-4274-b5f2-3ee017c8b682 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:00.097266987 +0000 UTC m=+34.258343449 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-n6b2h" (UniqueName: "kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h") pod "network-check-target-9hcgm" (UID: "8d2c16f2-642c-4274-b5f2-3ee017c8b682") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:18:44.397583 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:44.397507 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:44.397756 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:44.397521 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:44.397756 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:44.397650 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwpln" podUID="69592465-7d48-4eda-9662-361f7386ea86" Jul 14 11:18:44.397756 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:44.397734 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hcgm" podUID="8d2c16f2-642c-4274-b5f2-3ee017c8b682" Jul 14 11:18:45.398231 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:45.398191 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:45.398778 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:45.398319 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-882pd" podUID="0fc0a645-d7c2-459a-8ecb-b0bae335b9ca" Jul 14 11:18:46.398232 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:46.398188 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:46.398416 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:46.398311 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwpln" podUID="69592465-7d48-4eda-9662-361f7386ea86" Jul 14 11:18:46.398416 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:46.398398 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:46.398848 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:46.398489 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hcgm" podUID="8d2c16f2-642c-4274-b5f2-3ee017c8b682" Jul 14 11:18:47.398251 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:47.398225 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:47.398396 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:47.398333 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-882pd" podUID="0fc0a645-d7c2-459a-8ecb-b0bae335b9ca" Jul 14 11:18:48.398490 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.398224 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:48.399258 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.398278 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:48.399258 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:48.398570 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwpln" podUID="69592465-7d48-4eda-9662-361f7386ea86" Jul 14 11:18:48.399258 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:48.398625 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hcgm" podUID="8d2c16f2-642c-4274-b5f2-3ee017c8b682" Jul 14 11:18:48.493669 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.493617 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-62zzs" event={"ID":"3a64b363-5cc2-41ff-8a37-3df9b572ea88","Type":"ContainerStarted","Data":"4a2a33516f80dece63593c5b346b2a3b9e0b01a359ebc3135259f185e2075ac6"} Jul 14 11:18:48.496461 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.496430 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" event={"ID":"13f756c6-5b03-4308-8798-b0c75cc39c3b","Type":"ContainerStarted","Data":"bf35a0bcab37925941ac5e49ddb47d0c449bca81a5c9a07fee91bbec31955819"} Jul 14 11:18:48.499264 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.499245 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 11:18:48.499649 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.499603 2571 generic.go:356] "Generic (PLEG): container finished" podID="5baf7735-3ad0-40de-b92f-1b77fc61c592" containerID="2bc29a8e6fe6937c9ee081c4358f5bb46cecae2cdf7fa7b8ae806830daed6a7d" exitCode=1 Jul 14 11:18:48.499745 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.499651 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" event={"ID":"5baf7735-3ad0-40de-b92f-1b77fc61c592","Type":"ContainerStarted","Data":"6cc01cd5cabd877ddc1d9168947ea57cecf1ab5c17bbe52fe45a44d71b6c786a"} Jul 14 11:18:48.499745 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.499721 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" event={"ID":"5baf7735-3ad0-40de-b92f-1b77fc61c592","Type":"ContainerStarted","Data":"26a7809c943ec607eb1d2f767f485094bc831b427f53085f2ff8822470e2c900"} Jul 14 11:18:48.499858 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.499765 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" event={"ID":"5baf7735-3ad0-40de-b92f-1b77fc61c592","Type":"ContainerStarted","Data":"00df00b194a3063aaa14c40eb44741229d4c41caab01b49db7fffb0b8733cf42"} Jul 14 11:18:48.499858 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.499809 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" event={"ID":"5baf7735-3ad0-40de-b92f-1b77fc61c592","Type":"ContainerDied","Data":"2bc29a8e6fe6937c9ee081c4358f5bb46cecae2cdf7fa7b8ae806830daed6a7d"} Jul 14 11:18:48.499858 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.499846 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" event={"ID":"5baf7735-3ad0-40de-b92f-1b77fc61c592","Type":"ContainerStarted","Data":"e98b79a83e61c11ed5eb44053ea01ef12d5e08f0df1f7c5f000e294df182bfd1"} Jul 14 11:18:48.501192 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.501153 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8m4rg" event={"ID":"e67c4044-1964-428c-8a91-82e48aec2b66","Type":"ContainerStarted","Data":"239e941c654f1791b496a36abd9e43c97f029c447f7a35748d67a965e35f7b7f"} Jul 14 11:18:48.502641 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.502612 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ttsdl" event={"ID":"fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a","Type":"ContainerStarted","Data":"c589a1d155725e23432d1349188a394ceaf31814ccaaeac28f2bff2d95298d35"} Jul 14 11:18:48.504313 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.504289 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zmxv2" event={"ID":"b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4","Type":"ContainerStarted","Data":"b43f03b632827e9901c118a537831c6176e22f7dac8b1ddb694fa742cbd31a3f"} Jul 14 11:18:48.506094 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.506072 2571 generic.go:356] "Generic (PLEG): container finished" podID="b7040e83-67d0-46ef-a022-91faba1e83b2" containerID="fb542000b0d23742436052daa80e437cd7a89b3a54674355adb403e36f786a1d" exitCode=0 Jul 14 11:18:48.506198 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.506153 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7784t" event={"ID":"b7040e83-67d0-46ef-a022-91faba1e83b2","Type":"ContainerDied","Data":"fb542000b0d23742436052daa80e437cd7a89b3a54674355adb403e36f786a1d"} Jul 14 11:18:48.508263 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.508243 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" event={"ID":"11a2bd40-f5eb-4981-8e86-4ec6014436ea","Type":"ContainerStarted","Data":"39b55952f083ea3fcb79a08f381d5b18c530796a47b6db121d1b3abc7f7d5e29"} Jul 14 11:18:48.510448 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.510400 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" podStartSLOduration=21.510384177 podStartE2EDuration="21.510384177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:31.48724101 +0000 UTC m=+5.648317479" watchObservedRunningTime="2026-07-14 11:18:48.510384177 +0000 UTC m=+22.671460645" Jul 14 11:18:48.530337 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.530283 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zp5kb" podStartSLOduration=4.0229658 podStartE2EDuration="22.530267724s" totalImagesPullingTime="18.507301924s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:48.530060641 +0000 UTC m=+22.691137108" watchObservedRunningTime="2026-07-14 11:18:48.530267724 +0000 UTC m=+22.691344192" Jul 14 11:18:48.530790 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.530759 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-62zzs" podStartSLOduration=8.392963739 podStartE2EDuration="22.530751717s" totalImagesPullingTime="14.137787978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:48.51010633 +0000 UTC m=+22.671182798" watchObservedRunningTime="2026-07-14 11:18:48.530751717 +0000 UTC m=+22.691828251" Jul 14 11:18:48.545012 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.544955 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ttsdl" podStartSLOduration=4.054271403 podStartE2EDuration="22.544936673s" totalImagesPullingTime="18.49066527s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:48.544893404 +0000 UTC m=+22.705969872" watchObservedRunningTime="2026-07-14 11:18:48.544936673 +0000 UTC m=+22.706013142" Jul 14 11:18:48.566150 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.566109 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8m4rg" podStartSLOduration=4.00710439 podStartE2EDuration="22.566095371s" totalImagesPullingTime="18.558990981s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:48.565616657 +0000 UTC m=+22.726693126" watchObservedRunningTime="2026-07-14 11:18:48.566095371 +0000 UTC m=+22.727171837" Jul 14 11:18:48.580764 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.580716 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zmxv2" podStartSLOduration=12.11136948 podStartE2EDuration="22.580697597s" totalImagesPullingTime="10.469328117s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:48.580312786 +0000 UTC m=+22.741389255" watchObservedRunningTime="2026-07-14 11:18:48.580697597 +0000 UTC m=+22.741774066" Jul 14 11:18:48.719505 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.719436 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:48.795208 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.795175 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:48.869352 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:48.869323 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:49.034819 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:49.034752 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:49.117228 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:49.117198 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:49.212848 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:49.212821 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:49.261903 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:49.261879 2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 14 11:18:49.321667 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:49.321475 2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T11:18:49.261899268Z","UUID":"2c07c1cf-7045-44ae-a005-039788158a75","Handler":null,"Name":"","Endpoint":""} Jul 14 11:18:49.323532 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:49.323511 2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 14 11:18:49.323669 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:49.323557 2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 14 11:18:49.398211 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:49.398172 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:49.398370 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:49.398301 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-882pd" podUID="0fc0a645-d7c2-459a-8ecb-b0bae335b9ca" Jul 14 11:18:49.513072 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:49.513036 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" event={"ID":"13f756c6-5b03-4308-8798-b0c75cc39c3b","Type":"ContainerStarted","Data":"bb28c0a62278e706d735930f344031cf5d50adedea7688ebbcf241570a4d016d"} Jul 14 11:18:49.516220 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:49.516201 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 11:18:49.516639 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:49.516594 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" event={"ID":"5baf7735-3ad0-40de-b92f-1b77fc61c592","Type":"ContainerStarted","Data":"54983cb90e512688d4f5b70087b98e29fbe43ca707f40b3a98cfdf2e22eaa673"} Jul 14 11:18:49.518136 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:49.518112 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-w9vw6" event={"ID":"5be9ddca-55fc-4bc9-8a7a-aa245596c785","Type":"ContainerStarted","Data":"b54908761e98982436153d9f0f1cf0044b22ef7fb92865f6f8c07859c700ba16"} Jul 14 11:18:49.533807 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:49.533760 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-w9vw6" podStartSLOduration=5.0395269129999996 podStartE2EDuration="23.533739352s" totalImagesPullingTime="18.494212439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:49.533352039 +0000 UTC m=+23.694428507" watchObservedRunningTime="2026-07-14 11:18:49.533739352 +0000 UTC m=+23.694815821" Jul 14 11:18:49.655428 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:49.655331 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:50.029333 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:50.029260 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:50.129874 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:50.129817 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:50.398412 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:50.398325 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:50.398570 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:50.398325 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:50.398570 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:50.398476 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwpln" podUID="69592465-7d48-4eda-9662-361f7386ea86" Jul 14 11:18:50.398570 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:50.398554 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hcgm" podUID="8d2c16f2-642c-4274-b5f2-3ee017c8b682" Jul 14 11:18:51.045237 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:51.045004 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:51.147131 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:51.147083 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:51.284101 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:51.284064 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:51.383203 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:51.383125 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:51.398242 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:51.398214 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:51.398429 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:51.398342 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-882pd" podUID="0fc0a645-d7c2-459a-8ecb-b0bae335b9ca" Jul 14 11:18:51.525529 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:51.525490 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" event={"ID":"13f756c6-5b03-4308-8798-b0c75cc39c3b","Type":"ContainerStarted","Data":"0ec295d58d42a750ce92306c75119c2e5e055b5b0de2b4475b0cdb314900966b"} Jul 14 11:18:51.528849 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:51.528825 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 11:18:51.529274 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:51.529252 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" event={"ID":"5baf7735-3ad0-40de-b92f-1b77fc61c592","Type":"ContainerStarted","Data":"73d49a3df8532e21757900a8b031b76e46ac05c383a7b330ccd7f8af8082eca4"} Jul 14 11:18:51.544724 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:51.544663 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9gg9c" podStartSLOduration=4.752932402 podStartE2EDuration="25.544644607s" totalImagesPullingTime="20.791712205s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:51.544514585 +0000 UTC m=+25.705591052" watchObservedRunningTime="2026-07-14 11:18:51.544644607 +0000 UTC m=+25.705721067" Jul 14 11:18:52.034252 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:52.034219 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:52.138168 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:52.138133 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:52.397709 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:52.397618 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:52.397709 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:52.397664 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:52.397909 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:52.397769 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwpln" podUID="69592465-7d48-4eda-9662-361f7386ea86" Jul 14 11:18:52.397909 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:52.397885 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hcgm" podUID="8d2c16f2-642c-4274-b5f2-3ee017c8b682" Jul 14 11:18:52.429644 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:52.429597 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:52.492016 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:52.491981 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:52.634442 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:52.634407 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-62zzs" Jul 14 11:18:52.635265 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:52.635244 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-62zzs" Jul 14 11:18:52.990378 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:52.990117 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:53.398066 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:53.398027 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:53.398670 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:53.398122 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-882pd" podUID="0fc0a645-d7c2-459a-8ecb-b0bae335b9ca" Jul 14 11:18:53.536699 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:53.536675 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 11:18:53.537101 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:53.537073 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" event={"ID":"5baf7735-3ad0-40de-b92f-1b77fc61c592","Type":"ContainerStarted","Data":"bf99f7f8ce376ba1eddbc28f2eafcb30aac4e12da45167df3825b8a7ef25b98c"} Jul 14 11:18:53.537323 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:53.537297 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:53.537617 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:53.537603 2571 scope.go:117] "RemoveContainer" containerID="2bc29a8e6fe6937c9ee081c4358f5bb46cecae2cdf7fa7b8ae806830daed6a7d" Jul 14 11:18:53.538989 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:53.538964 2571 generic.go:356] "Generic (PLEG): container finished" podID="b7040e83-67d0-46ef-a022-91faba1e83b2" containerID="8f02eda08d959b6b374c4a5d166303076a1c686e89ccfb68b7aedd008aa35c98" exitCode=0 Jul 14 11:18:53.539068 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:53.539040 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7784t" event={"ID":"b7040e83-67d0-46ef-a022-91faba1e83b2","Type":"ContainerDied","Data":"8f02eda08d959b6b374c4a5d166303076a1c686e89ccfb68b7aedd008aa35c98"} Jul 14 11:18:53.539263 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:53.539242 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-62zzs" Jul 14 11:18:53.539808 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:53.539781 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-62zzs" Jul 14 11:18:53.554710 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:53.554692 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:53.657767 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:53.657699 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:53.827530 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:53.827502 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:53.897292 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:53.897265 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:54.397973 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:54.397892 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:54.398159 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:54.398021 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwpln" podUID="69592465-7d48-4eda-9662-361f7386ea86" Jul 14 11:18:54.398159 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:54.398071 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:54.398523 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:54.398157 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hcgm" podUID="8d2c16f2-642c-4274-b5f2-3ee017c8b682" Jul 14 11:18:54.544314 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:54.544290 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 11:18:54.544729 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:54.544696 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" event={"ID":"5baf7735-3ad0-40de-b92f-1b77fc61c592","Type":"ContainerStarted","Data":"d18e036f1876a4eca4ed9d53b8a2301191ff9df02c14d18635b3b8bb866ee066"} Jul 14 11:18:54.544819 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:54.544768 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:54.544908 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:54.544876 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:54.560490 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:54.560465 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:18:54.574998 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:54.574941 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" podStartSLOduration=10.005058154 podStartE2EDuration="28.574925386s" totalImagesPullingTime="18.569867232s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:18:54.57319778 +0000 UTC m=+28.734274259" watchObservedRunningTime="2026-07-14 11:18:54.574925386 +0000 UTC m=+28.736001853" Jul 14 11:18:54.879474 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:54.879269 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fwpln"] Jul 14 11:18:54.879608 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:54.879593 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:54.879773 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:54.879737 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwpln" podUID="69592465-7d48-4eda-9662-361f7386ea86" Jul 14 11:18:54.882268 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:54.882247 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-882pd"] Jul 14 11:18:54.882367 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:54.882358 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:54.882480 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:54.882442 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-882pd" podUID="0fc0a645-d7c2-459a-8ecb-b0bae335b9ca" Jul 14 11:18:54.882887 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:54.882869 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9hcgm"] Jul 14 11:18:54.882968 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:54.882944 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:54.883027 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:54.883009 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hcgm" podUID="8d2c16f2-642c-4274-b5f2-3ee017c8b682" Jul 14 11:18:55.177160 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:55.177133 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ttsdl_fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a/dns-node-resolver/0.log" Jul 14 11:18:55.548643 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:55.548544 2571 generic.go:356] "Generic (PLEG): container finished" podID="b7040e83-67d0-46ef-a022-91faba1e83b2" containerID="a6c9d27cb3be224339a0675a8b08099d054bc5039a30359690cbc13da0e1dafa" exitCode=0 Jul 14 11:18:55.549290 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:55.548648 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7784t" event={"ID":"b7040e83-67d0-46ef-a022-91faba1e83b2","Type":"ContainerDied","Data":"a6c9d27cb3be224339a0675a8b08099d054bc5039a30359690cbc13da0e1dafa"} Jul 14 11:18:56.176385 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:56.176361 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zmxv2_b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4/node-ca/0.log" Jul 14 11:18:56.398369 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:56.398341 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:56.398553 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:56.398395 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:56.398553 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:56.398459 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-882pd" podUID="0fc0a645-d7c2-459a-8ecb-b0bae335b9ca" Jul 14 11:18:56.398553 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:56.398490 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:56.398733 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:56.398591 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hcgm" podUID="8d2c16f2-642c-4274-b5f2-3ee017c8b682" Jul 14 11:18:56.398733 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:56.398701 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwpln" podUID="69592465-7d48-4eda-9662-361f7386ea86" Jul 14 11:18:56.952135 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:56.952107 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:57.057696 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:57.057663 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:57.151959 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:57.151928 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:57.555045 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:57.555013 2571 generic.go:356] "Generic (PLEG): container finished" podID="b7040e83-67d0-46ef-a022-91faba1e83b2" containerID="293f2b6375da572d0c2475f0262f3c2bbabb38aba7870d24a018f8f950134952" exitCode=0 Jul 14 11:18:57.555178 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:57.555078 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7784t" event={"ID":"b7040e83-67d0-46ef-a022-91faba1e83b2","Type":"ContainerDied","Data":"293f2b6375da572d0c2475f0262f3c2bbabb38aba7870d24a018f8f950134952"} Jul 14 11:18:58.398025 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:58.397986 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:18:58.398025 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:58.398026 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:18:58.398713 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:58.398039 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:18:58.398713 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:58.398128 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hcgm" podUID="8d2c16f2-642c-4274-b5f2-3ee017c8b682" Jul 14 11:18:58.398713 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:58.398212 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-882pd" podUID="0fc0a645-d7c2-459a-8ecb-b0bae335b9ca" Jul 14 11:18:58.398713 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:18:58.398333 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwpln" podUID="69592465-7d48-4eda-9662-361f7386ea86" Jul 14 11:18:59.596582 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:59.596544 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:18:59.670506 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:18:59.670470 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:00.124005 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.123966 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs\") pod \"network-metrics-daemon-fwpln\" (UID: \"69592465-7d48-4eda-9662-361f7386ea86\") " pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:19:00.124187 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.124016 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6b2h\" (UniqueName: \"kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h\") pod \"network-check-target-9hcgm\" (UID: \"8d2c16f2-642c-4274-b5f2-3ee017c8b682\") " pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:19:00.124187 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.124043 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret\") pod \"global-pull-secret-syncer-882pd\" (UID: \"0fc0a645-d7c2-459a-8ecb-b0bae335b9ca\") " pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:19:00.124187 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:00.124120 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:19:00.124187 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:00.124186 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 11:19:00.124369 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:00.124201 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 11:19:00.124369 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:00.124226 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 11:19:00.124369 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:00.124188 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs podName:69592465-7d48-4eda-9662-361f7386ea86 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:32.124169247 +0000 UTC m=+66.285245709 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs") pod "network-metrics-daemon-fwpln" (UID: "69592465-7d48-4eda-9662-361f7386ea86") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 11:19:00.124369 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:00.124240 2571 projected.go:196] Error preparing data for projected volume kube-api-access-n6b2h for pod openshift-network-diagnostics/network-check-target-9hcgm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:19:00.124369 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:00.124273 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret podName:0fc0a645-d7c2-459a-8ecb-b0bae335b9ca nodeName:}" failed. No retries permitted until 2026-07-14 11:19:32.124246438 +0000 UTC m=+66.285322886 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret") pod "global-pull-secret-syncer-882pd" (UID: "0fc0a645-d7c2-459a-8ecb-b0bae335b9ca") : object "kube-system"/"original-pull-secret" not registered Jul 14 11:19:00.124369 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:00.124291 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h podName:8d2c16f2-642c-4274-b5f2-3ee017c8b682 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:32.124281891 +0000 UTC m=+66.285358338 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-n6b2h" (UniqueName: "kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h") pod "network-check-target-9hcgm" (UID: "8d2c16f2-642c-4274-b5f2-3ee017c8b682") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 11:19:00.291116 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.291083 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:00.393109 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.393078 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:00.397405 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.397378 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:19:00.397551 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.397378 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:19:00.397551 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.397512 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:19:00.397551 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:00.397509 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fwpln" podUID="69592465-7d48-4eda-9662-361f7386ea86" Jul 14 11:19:00.397754 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:00.397598 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9hcgm" podUID="8d2c16f2-642c-4274-b5f2-3ee017c8b682" Jul 14 11:19:00.397754 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:00.397716 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-882pd" podUID="0fc0a645-d7c2-459a-8ecb-b0bae335b9ca" Jul 14 11:19:00.687577 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.687498 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeReady" Jul 14 11:19:00.687994 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.687658 2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 14 11:19:00.723502 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.723436 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-559449c858-hbtxv"] Jul 14 11:19:00.756584 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.756557 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rc2bq"] Jul 14 11:19:00.756795 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.756747 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:00.760816 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.760786 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rhbkm\"" Jul 14 11:19:00.761002 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.760969 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 14 11:19:00.761281 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.761259 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 14 11:19:00.762439 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.762417 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 14 11:19:00.767208 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.767187 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 14 11:19:00.768782 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.768765 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-z9xv5"] Jul 14 11:19:00.768900 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.768880 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:00.771860 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.771702 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 14 11:19:00.771860 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.771735 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 14 11:19:00.771860 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.771784 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 14 11:19:00.771860 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.771822 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xrcwt\"" Jul 14 11:19:00.772088 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.771993 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 14 11:19:00.790814 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.790788 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-559449c858-hbtxv"] Jul 14 11:19:00.790945 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.790830 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rc2bq"] Jul 14 11:19:00.790945 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.790860 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z9xv5"] Jul 14 11:19:00.790945 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.790915 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z9xv5" Jul 14 11:19:00.793956 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.793931 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9c655\"" Jul 14 11:19:00.794056 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.793983 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 14 11:19:00.794056 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.794010 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 14 11:19:00.794056 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.794030 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 14 11:19:00.832808 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.832769 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:00.838430 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.838408 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9lqh6"] Jul 14 11:19:00.860868 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.860841 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9lqh6"] Jul 14 11:19:00.861025 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.860932 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9lqh6" Jul 14 11:19:00.863821 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.863799 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 14 11:19:00.863941 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.863851 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 14 11:19:00.863941 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.863858 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hgqwk\"" Jul 14 11:19:00.905091 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.905009 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:00.931817 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.931789 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e85830b4-be15-4f4c-ab66-62ecc464628f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:00.932000 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.931829 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/584baebf-0808-4f20-adad-c8685a1eb082-image-registry-private-configuration\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:00.932000 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.931848 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/584baebf-0808-4f20-adad-c8685a1eb082-ca-trust-extracted\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:00.932000 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.931886 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-registry-tls\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:00.932000 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.931910 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/584baebf-0808-4f20-adad-c8685a1eb082-registry-certificates\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:00.932182 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.932057 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbt7b\" (UniqueName: \"kubernetes.io/projected/e142c698-e9cd-4a19-943f-74132dd73b2f-kube-api-access-nbt7b\") pod \"ingress-canary-z9xv5\" (UID: \"e142c698-e9cd-4a19-943f-74132dd73b2f\") " pod="openshift-ingress-canary/ingress-canary-z9xv5" Jul 14 11:19:00.932182 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.932106 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-bound-sa-token\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:00.932182 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.932153 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e85830b4-be15-4f4c-ab66-62ecc464628f-crio-socket\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:00.932182 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.932177 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e85830b4-be15-4f4c-ab66-62ecc464628f-data-volume\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:00.932352 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.932199 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e85830b4-be15-4f4c-ab66-62ecc464628f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:00.932352 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.932225 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpzvj\" (UniqueName: \"kubernetes.io/projected/e85830b4-be15-4f4c-ab66-62ecc464628f-kube-api-access-xpzvj\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:00.932352 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.932294 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e142c698-e9cd-4a19-943f-74132dd73b2f-cert\") pod \"ingress-canary-z9xv5\" (UID: \"e142c698-e9cd-4a19-943f-74132dd73b2f\") " pod="openshift-ingress-canary/ingress-canary-z9xv5" Jul 14 11:19:00.932467 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.932369 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/584baebf-0808-4f20-adad-c8685a1eb082-trusted-ca\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:00.932467 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.932397 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/584baebf-0808-4f20-adad-c8685a1eb082-installation-pull-secrets\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:00.932467 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:00.932423 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kr97\" (UniqueName: \"kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-kube-api-access-4kr97\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:01.008328 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.008250 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:01.033251 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.033218 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-tmp-dir\") pod \"dns-default-9lqh6\" (UID: \"6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5\") " pod="openshift-dns/dns-default-9lqh6" Jul 14 11:19:01.033428 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.033261 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-registry-tls\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:01.033428 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.033281 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/584baebf-0808-4f20-adad-c8685a1eb082-registry-certificates\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:01.033428 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.033322 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-metrics-tls\") pod \"dns-default-9lqh6\" (UID: \"6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5\") " pod="openshift-dns/dns-default-9lqh6" Jul 14 11:19:01.033428 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.033355 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgtc6\" (UniqueName: \"kubernetes.io/projected/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-kube-api-access-fgtc6\") pod \"dns-default-9lqh6\" (UID: \"6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5\") " pod="openshift-dns/dns-default-9lqh6" Jul 14 11:19:01.033428 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.033395 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbt7b\" (UniqueName: \"kubernetes.io/projected/e142c698-e9cd-4a19-943f-74132dd73b2f-kube-api-access-nbt7b\") pod \"ingress-canary-z9xv5\" (UID: \"e142c698-e9cd-4a19-943f-74132dd73b2f\") " pod="openshift-ingress-canary/ingress-canary-z9xv5" Jul 14 11:19:01.033428 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.033420 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 11:19:01.033755 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.033441 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559449c858-hbtxv: secret "image-registry-tls" not found Jul 14 11:19:01.033755 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.033438 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-bound-sa-token\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:01.033755 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.033552 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-registry-tls podName:584baebf-0808-4f20-adad-c8685a1eb082 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:01.53352943 +0000 UTC m=+35.694605894 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-registry-tls") pod "image-registry-559449c858-hbtxv" (UID: "584baebf-0808-4f20-adad-c8685a1eb082") : secret "image-registry-tls" not found Jul 14 11:19:01.033755 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.033611 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e85830b4-be15-4f4c-ab66-62ecc464628f-crio-socket\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:01.033755 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.033660 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e85830b4-be15-4f4c-ab66-62ecc464628f-data-volume\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:01.033755 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.033686 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e85830b4-be15-4f4c-ab66-62ecc464628f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:01.033755 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.033714 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpzvj\" (UniqueName: \"kubernetes.io/projected/e85830b4-be15-4f4c-ab66-62ecc464628f-kube-api-access-xpzvj\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:01.033755 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.033743 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e142c698-e9cd-4a19-943f-74132dd73b2f-cert\") pod \"ingress-canary-z9xv5\" (UID: \"e142c698-e9cd-4a19-943f-74132dd73b2f\") " pod="openshift-ingress-canary/ingress-canary-z9xv5" Jul 14 11:19:01.034134 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.033777 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/584baebf-0808-4f20-adad-c8685a1eb082-trusted-ca\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:01.034134 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.033795 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 11:19:01.034134 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.033802 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/584baebf-0808-4f20-adad-c8685a1eb082-installation-pull-secrets\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:01.034134 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.033827 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kr97\" (UniqueName: \"kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-kube-api-access-4kr97\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:01.034134 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.033836 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e85830b4-be15-4f4c-ab66-62ecc464628f-crio-socket\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:01.034134 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.033848 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e85830b4-be15-4f4c-ab66-62ecc464628f-insights-runtime-extractor-tls podName:e85830b4-be15-4f4c-ab66-62ecc464628f nodeName:}" failed. No retries permitted until 2026-07-14 11:19:01.533831946 +0000 UTC m=+35.694908392 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e85830b4-be15-4f4c-ab66-62ecc464628f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rc2bq" (UID: "e85830b4-be15-4f4c-ab66-62ecc464628f") : secret "insights-runtime-extractor-tls" not found Jul 14 11:19:01.034134 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.033911 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e85830b4-be15-4f4c-ab66-62ecc464628f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:01.034134 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.033939 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-config-volume\") pod \"dns-default-9lqh6\" (UID: \"6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5\") " pod="openshift-dns/dns-default-9lqh6" Jul 14 11:19:01.034134 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.033972 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/584baebf-0808-4f20-adad-c8685a1eb082-image-registry-private-configuration\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:01.034134 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.033991 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e85830b4-be15-4f4c-ab66-62ecc464628f-data-volume\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:01.034134 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.034007 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/584baebf-0808-4f20-adad-c8685a1eb082-ca-trust-extracted\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:01.034134 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.034029 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/584baebf-0808-4f20-adad-c8685a1eb082-registry-certificates\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:01.034666 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.034216 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 11:19:01.034666 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.034288 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e85830b4-be15-4f4c-ab66-62ecc464628f-kube-rbac-proxy-cm podName:e85830b4-be15-4f4c-ab66-62ecc464628f nodeName:}" failed. No retries permitted until 2026-07-14 11:19:01.534271649 +0000 UTC m=+35.695348097 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e85830b4-be15-4f4c-ab66-62ecc464628f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rc2bq" (UID: "e85830b4-be15-4f4c-ab66-62ecc464628f") : configmap "kube-rbac-proxy" not found Jul 14 11:19:01.034666 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.034565 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/584baebf-0808-4f20-adad-c8685a1eb082-ca-trust-extracted\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:01.034820 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.034706 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/584baebf-0808-4f20-adad-c8685a1eb082-trusted-ca\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:01.038725 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.038694 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/584baebf-0808-4f20-adad-c8685a1eb082-image-registry-private-configuration\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:01.038725 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.038712 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/584baebf-0808-4f20-adad-c8685a1eb082-installation-pull-secrets\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:01.038909 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.038709 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e142c698-e9cd-4a19-943f-74132dd73b2f-cert\") pod \"ingress-canary-z9xv5\" (UID: \"e142c698-e9cd-4a19-943f-74132dd73b2f\") " pod="openshift-ingress-canary/ingress-canary-z9xv5" Jul 14 11:19:01.043031 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.042992 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-bound-sa-token\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:01.043351 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.043329 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpzvj\" (UniqueName: \"kubernetes.io/projected/e85830b4-be15-4f4c-ab66-62ecc464628f-kube-api-access-xpzvj\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:01.043831 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.043804 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbt7b\" (UniqueName: \"kubernetes.io/projected/e142c698-e9cd-4a19-943f-74132dd73b2f-kube-api-access-nbt7b\") pod \"ingress-canary-z9xv5\" (UID: \"e142c698-e9cd-4a19-943f-74132dd73b2f\") " pod="openshift-ingress-canary/ingress-canary-z9xv5" Jul 14 11:19:01.043952 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.043941 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kr97\" (UniqueName: \"kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-kube-api-access-4kr97\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:01.057068 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.057047 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-g6pxd"] Jul 14 11:19:01.087082 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.087055 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-g6pxd"] Jul 14 11:19:01.087227 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.087148 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-g6pxd" Jul 14 11:19:01.090616 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.090592 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 14 11:19:01.090749 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.090669 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 14 11:19:01.090749 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.090744 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-s9hqx\"" Jul 14 11:19:01.101204 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.101184 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z9xv5" Jul 14 11:19:01.135063 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.135031 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgtc6\" (UniqueName: \"kubernetes.io/projected/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-kube-api-access-fgtc6\") pod \"dns-default-9lqh6\" (UID: \"6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5\") " pod="openshift-dns/dns-default-9lqh6" Jul 14 11:19:01.135192 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.135159 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-config-volume\") pod \"dns-default-9lqh6\" (UID: \"6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5\") " pod="openshift-dns/dns-default-9lqh6" Jul 14 11:19:01.135256 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.135192 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-tmp-dir\") pod \"dns-default-9lqh6\" (UID: \"6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5\") " pod="openshift-dns/dns-default-9lqh6" Jul 14 11:19:01.135256 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.135229 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-metrics-tls\") pod \"dns-default-9lqh6\" (UID: \"6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5\") " pod="openshift-dns/dns-default-9lqh6" Jul 14 11:19:01.135369 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.135324 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 11:19:01.135423 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.135385 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-metrics-tls podName:6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:01.635366933 +0000 UTC m=+35.796443381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-metrics-tls") pod "dns-default-9lqh6" (UID: "6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5") : secret "dns-default-metrics-tls" not found Jul 14 11:19:01.135601 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.135577 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-tmp-dir\") pod \"dns-default-9lqh6\" (UID: \"6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5\") " pod="openshift-dns/dns-default-9lqh6" Jul 14 11:19:01.135815 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.135790 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-config-volume\") pod \"dns-default-9lqh6\" (UID: \"6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5\") " pod="openshift-dns/dns-default-9lqh6" Jul 14 11:19:01.144859 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.144839 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgtc6\" (UniqueName: \"kubernetes.io/projected/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-kube-api-access-fgtc6\") pod \"dns-default-9lqh6\" (UID: \"6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5\") " pod="openshift-dns/dns-default-9lqh6" Jul 14 11:19:01.236554 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.236518 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/266bd207-c148-4d87-8eb7-baa4f02aa433-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-g6pxd\" (UID: \"266bd207-c148-4d87-8eb7-baa4f02aa433\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-g6pxd" Jul 14 11:19:01.236750 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.236650 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/266bd207-c148-4d87-8eb7-baa4f02aa433-nginx-conf\") pod \"networking-console-plugin-6cf856758c-g6pxd\" (UID: \"266bd207-c148-4d87-8eb7-baa4f02aa433\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-g6pxd" Jul 14 11:19:01.337216 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.337130 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/266bd207-c148-4d87-8eb7-baa4f02aa433-nginx-conf\") pod \"networking-console-plugin-6cf856758c-g6pxd\" (UID: \"266bd207-c148-4d87-8eb7-baa4f02aa433\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-g6pxd" Jul 14 11:19:01.337369 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.337230 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/266bd207-c148-4d87-8eb7-baa4f02aa433-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-g6pxd\" (UID: \"266bd207-c148-4d87-8eb7-baa4f02aa433\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-g6pxd" Jul 14 11:19:01.337369 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.337340 2571 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 11:19:01.337488 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.337399 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/266bd207-c148-4d87-8eb7-baa4f02aa433-networking-console-plugin-cert podName:266bd207-c148-4d87-8eb7-baa4f02aa433 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:01.837381352 +0000 UTC m=+35.998457820 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/266bd207-c148-4d87-8eb7-baa4f02aa433-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-g6pxd" (UID: "266bd207-c148-4d87-8eb7-baa4f02aa433") : secret "networking-console-plugin-cert" not found Jul 14 11:19:01.337872 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.337854 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/266bd207-c148-4d87-8eb7-baa4f02aa433-nginx-conf\") pod \"networking-console-plugin-6cf856758c-g6pxd\" (UID: \"266bd207-c148-4d87-8eb7-baa4f02aa433\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-g6pxd" Jul 14 11:19:01.538984 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.538943 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-registry-tls\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:01.539166 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.539093 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e85830b4-be15-4f4c-ab66-62ecc464628f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:01.539166 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.539156 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e85830b4-be15-4f4c-ab66-62ecc464628f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:01.539291 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.539168 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 11:19:01.539291 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.539192 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559449c858-hbtxv: secret "image-registry-tls" not found Jul 14 11:19:01.539291 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.539245 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 11:19:01.539291 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.539258 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-registry-tls podName:584baebf-0808-4f20-adad-c8685a1eb082 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:02.539236933 +0000 UTC m=+36.700313401 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-registry-tls") pod "image-registry-559449c858-hbtxv" (UID: "584baebf-0808-4f20-adad-c8685a1eb082") : secret "image-registry-tls" not found Jul 14 11:19:01.539291 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.539292 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e85830b4-be15-4f4c-ab66-62ecc464628f-kube-rbac-proxy-cm podName:e85830b4-be15-4f4c-ab66-62ecc464628f nodeName:}" failed. No retries permitted until 2026-07-14 11:19:02.539278961 +0000 UTC m=+36.700355406 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e85830b4-be15-4f4c-ab66-62ecc464628f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rc2bq" (UID: "e85830b4-be15-4f4c-ab66-62ecc464628f") : configmap "kube-rbac-proxy" not found Jul 14 11:19:01.539533 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.539332 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 11:19:01.539533 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.539400 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e85830b4-be15-4f4c-ab66-62ecc464628f-insights-runtime-extractor-tls podName:e85830b4-be15-4f4c-ab66-62ecc464628f nodeName:}" failed. No retries permitted until 2026-07-14 11:19:02.53938489 +0000 UTC m=+36.700461352 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e85830b4-be15-4f4c-ab66-62ecc464628f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rc2bq" (UID: "e85830b4-be15-4f4c-ab66-62ecc464628f") : secret "insights-runtime-extractor-tls" not found Jul 14 11:19:01.640168 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.640073 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-metrics-tls\") pod \"dns-default-9lqh6\" (UID: \"6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5\") " pod="openshift-dns/dns-default-9lqh6" Jul 14 11:19:01.640320 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.640240 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 11:19:01.640376 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.640323 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-metrics-tls podName:6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:02.640299651 +0000 UTC m=+36.801376100 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-metrics-tls") pod "dns-default-9lqh6" (UID: "6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5") : secret "dns-default-metrics-tls" not found Jul 14 11:19:01.700113 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.700074 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:01.841226 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.841179 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/266bd207-c148-4d87-8eb7-baa4f02aa433-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-g6pxd\" (UID: \"266bd207-c148-4d87-8eb7-baa4f02aa433\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-g6pxd" Jul 14 11:19:01.841416 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.841345 2571 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 11:19:01.841482 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:01.841432 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/266bd207-c148-4d87-8eb7-baa4f02aa433-networking-console-plugin-cert podName:266bd207-c148-4d87-8eb7-baa4f02aa433 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:02.841405929 +0000 UTC m=+37.002482374 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/266bd207-c148-4d87-8eb7-baa4f02aa433-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-g6pxd" (UID: "266bd207-c148-4d87-8eb7-baa4f02aa433") : secret "networking-console-plugin-cert" not found Jul 14 11:19:01.959518 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.959483 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z9xv5"] Jul 14 11:19:01.970502 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:01.970471 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:02.071665 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:19:02.071618 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode142c698_e9cd_4a19_943f_74132dd73b2f.slice/crio-d2a9bbb65a092e7f2d8a6aadaf96a1e44ab5931e03d7d922c569990733579f72 WatchSource:0}: Error finding container d2a9bbb65a092e7f2d8a6aadaf96a1e44ab5931e03d7d922c569990733579f72: Status 404 returned error can't find the container with id d2a9bbb65a092e7f2d8a6aadaf96a1e44ab5931e03d7d922c569990733579f72 Jul 14 11:19:02.398036 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:02.397843 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:19:02.398207 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:02.397843 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:19:02.398676 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:02.397918 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:19:02.401663 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:02.401644 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 11:19:02.402906 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:02.402890 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 11:19:02.402906 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:02.402897 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 11:19:02.403072 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:02.402896 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bm9fn\"" Jul 14 11:19:02.403072 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:02.402896 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q2rq4\"" Jul 14 11:19:02.403072 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:02.402897 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 11:19:02.546877 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:02.546841 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e85830b4-be15-4f4c-ab66-62ecc464628f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:02.547017 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:02.546899 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e85830b4-be15-4f4c-ab66-62ecc464628f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:02.547017 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:02.546934 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-registry-tls\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:02.547142 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:02.547027 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 11:19:02.547142 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:02.547057 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 11:19:02.547142 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:02.547094 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e85830b4-be15-4f4c-ab66-62ecc464628f-kube-rbac-proxy-cm podName:e85830b4-be15-4f4c-ab66-62ecc464628f nodeName:}" failed. No retries permitted until 2026-07-14 11:19:04.547077553 +0000 UTC m=+38.708153998 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e85830b4-be15-4f4c-ab66-62ecc464628f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rc2bq" (UID: "e85830b4-be15-4f4c-ab66-62ecc464628f") : configmap "kube-rbac-proxy" not found Jul 14 11:19:02.547142 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:02.547120 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e85830b4-be15-4f4c-ab66-62ecc464628f-insights-runtime-extractor-tls podName:e85830b4-be15-4f4c-ab66-62ecc464628f nodeName:}" failed. No retries permitted until 2026-07-14 11:19:04.547101097 +0000 UTC m=+38.708177563 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e85830b4-be15-4f4c-ab66-62ecc464628f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rc2bq" (UID: "e85830b4-be15-4f4c-ab66-62ecc464628f") : secret "insights-runtime-extractor-tls" not found Jul 14 11:19:02.547142 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:02.547030 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 11:19:02.547142 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:02.547141 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559449c858-hbtxv: secret "image-registry-tls" not found Jul 14 11:19:02.547347 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:02.547187 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-registry-tls podName:584baebf-0808-4f20-adad-c8685a1eb082 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:04.547174907 +0000 UTC m=+38.708251358 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-registry-tls") pod "image-registry-559449c858-hbtxv" (UID: "584baebf-0808-4f20-adad-c8685a1eb082") : secret "image-registry-tls" not found Jul 14 11:19:02.570047 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:02.570006 2571 generic.go:356] "Generic (PLEG): container finished" podID="b7040e83-67d0-46ef-a022-91faba1e83b2" containerID="cb0626f9cf58f14ab3e1d2299561d90df659351c3ae4cf60cad548d417c93538" exitCode=0 Jul 14 11:19:02.570198 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:02.570050 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7784t" event={"ID":"b7040e83-67d0-46ef-a022-91faba1e83b2","Type":"ContainerDied","Data":"cb0626f9cf58f14ab3e1d2299561d90df659351c3ae4cf60cad548d417c93538"} Jul 14 11:19:02.571390 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:02.571345 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z9xv5" event={"ID":"e142c698-e9cd-4a19-943f-74132dd73b2f","Type":"ContainerStarted","Data":"d2a9bbb65a092e7f2d8a6aadaf96a1e44ab5931e03d7d922c569990733579f72"} Jul 14 11:19:02.647432 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:02.647402 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-metrics-tls\") pod \"dns-default-9lqh6\" (UID: \"6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5\") " pod="openshift-dns/dns-default-9lqh6" Jul 14 11:19:02.647589 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:02.647573 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 11:19:02.647674 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:02.647656 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-metrics-tls podName:6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:04.647621647 +0000 UTC m=+38.808698112 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-metrics-tls") pod "dns-default-9lqh6" (UID: "6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5") : secret "dns-default-metrics-tls" not found Jul 14 11:19:02.849540 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:02.849502 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/266bd207-c148-4d87-8eb7-baa4f02aa433-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-g6pxd\" (UID: \"266bd207-c148-4d87-8eb7-baa4f02aa433\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-g6pxd" Jul 14 11:19:02.849996 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:02.849699 2571 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 11:19:02.849996 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:02.849775 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/266bd207-c148-4d87-8eb7-baa4f02aa433-networking-console-plugin-cert podName:266bd207-c148-4d87-8eb7-baa4f02aa433 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:04.84975531 +0000 UTC m=+39.010831776 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/266bd207-c148-4d87-8eb7-baa4f02aa433-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-g6pxd" (UID: "266bd207-c148-4d87-8eb7-baa4f02aa433") : secret "networking-console-plugin-cert" not found Jul 14 11:19:03.202235 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:03.202194 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:03.577137 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:03.577057 2571 generic.go:356] "Generic (PLEG): container finished" podID="b7040e83-67d0-46ef-a022-91faba1e83b2" containerID="e1ff027b05a7c68c0f8bea355be2050d97625b8138e4810120b3da6eac688a0b" exitCode=0 Jul 14 11:19:03.577137 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:03.577110 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7784t" event={"ID":"b7040e83-67d0-46ef-a022-91faba1e83b2","Type":"ContainerDied","Data":"e1ff027b05a7c68c0f8bea355be2050d97625b8138e4810120b3da6eac688a0b"} Jul 14 11:19:04.563933 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:04.563904 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e85830b4-be15-4f4c-ab66-62ecc464628f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:04.564503 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:04.563948 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e85830b4-be15-4f4c-ab66-62ecc464628f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:04.564503 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:04.563972 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-registry-tls\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:04.564503 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:04.564057 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 11:19:04.564503 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:04.564075 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 11:19:04.564503 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:04.564131 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e85830b4-be15-4f4c-ab66-62ecc464628f-insights-runtime-extractor-tls podName:e85830b4-be15-4f4c-ab66-62ecc464628f nodeName:}" failed. No retries permitted until 2026-07-14 11:19:08.564115927 +0000 UTC m=+42.725192373 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e85830b4-be15-4f4c-ab66-62ecc464628f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rc2bq" (UID: "e85830b4-be15-4f4c-ab66-62ecc464628f") : secret "insights-runtime-extractor-tls" not found Jul 14 11:19:04.564503 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:04.564066 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 11:19:04.564503 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:04.564161 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559449c858-hbtxv: secret "image-registry-tls" not found Jul 14 11:19:04.564503 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:04.564162 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e85830b4-be15-4f4c-ab66-62ecc464628f-kube-rbac-proxy-cm podName:e85830b4-be15-4f4c-ab66-62ecc464628f nodeName:}" failed. No retries permitted until 2026-07-14 11:19:08.564143756 +0000 UTC m=+42.725220212 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e85830b4-be15-4f4c-ab66-62ecc464628f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rc2bq" (UID: "e85830b4-be15-4f4c-ab66-62ecc464628f") : configmap "kube-rbac-proxy" not found Jul 14 11:19:04.564503 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:04.564202 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-registry-tls podName:584baebf-0808-4f20-adad-c8685a1eb082 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:08.56419256 +0000 UTC m=+42.725269006 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-registry-tls") pod "image-registry-559449c858-hbtxv" (UID: "584baebf-0808-4f20-adad-c8685a1eb082") : secret "image-registry-tls" not found Jul 14 11:19:04.583589 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:04.583558 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7784t" event={"ID":"b7040e83-67d0-46ef-a022-91faba1e83b2","Type":"ContainerStarted","Data":"1bb722eca2e423ae1ac532e532464fee06204c8d6081090b1f2e97b9fe7c28f1"} Jul 14 11:19:04.585061 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:04.585038 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z9xv5" event={"ID":"e142c698-e9cd-4a19-943f-74132dd73b2f","Type":"ContainerStarted","Data":"90447024890b8683cf0674065922c30b674e007af9219eb601130a1895da3195"} Jul 14 11:19:04.609580 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:04.609495 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7784t" podStartSLOduration=9.021067844 podStartE2EDuration="38.60947899s" totalImagesPullingTime="29.588411146s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:04.607430782 +0000 UTC m=+38.768507248" watchObservedRunningTime="2026-07-14 11:19:04.60947899 +0000 UTC m=+38.770555456" Jul 14 11:19:04.623706 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:04.623667 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-z9xv5" podStartSLOduration=2.360332175 podStartE2EDuration="4.623657637s" totalImagesPullingTime="2.263325462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:19:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:04.623174744 +0000 UTC m=+38.784251212" watchObservedRunningTime="2026-07-14 11:19:04.623657637 +0000 UTC m=+38.784734104" Jul 14 11:19:04.664978 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:04.664952 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-metrics-tls\") pod \"dns-default-9lqh6\" (UID: \"6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5\") " pod="openshift-dns/dns-default-9lqh6" Jul 14 11:19:04.665124 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:04.665106 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 11:19:04.665187 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:04.665166 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-metrics-tls podName:6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:08.665152227 +0000 UTC m=+42.826228673 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-metrics-tls") pod "dns-default-9lqh6" (UID: "6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5") : secret "dns-default-metrics-tls" not found Jul 14 11:19:04.867132 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:04.867045 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/266bd207-c148-4d87-8eb7-baa4f02aa433-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-g6pxd\" (UID: \"266bd207-c148-4d87-8eb7-baa4f02aa433\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-g6pxd" Jul 14 11:19:04.867252 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:04.867180 2571 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 11:19:04.867252 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:04.867242 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/266bd207-c148-4d87-8eb7-baa4f02aa433-networking-console-plugin-cert podName:266bd207-c148-4d87-8eb7-baa4f02aa433 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:08.867225832 +0000 UTC m=+43.028302278 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/266bd207-c148-4d87-8eb7-baa4f02aa433-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-g6pxd" (UID: "266bd207-c148-4d87-8eb7-baa4f02aa433") : secret "networking-console-plugin-cert" not found Jul 14 11:19:05.620550 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:05.620517 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:06.318366 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:06.318336 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:06.995122 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:06.995093 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:07.094623 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:07.094593 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:07.196027 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:07.195997 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:07.301810 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:07.301724 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:07.413413 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:07.413381 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:07.522437 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:07.522407 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:07.621068 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:07.620959 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:07.825434 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:07.825406 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:08.595385 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:08.595352 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e85830b4-be15-4f4c-ab66-62ecc464628f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:08.595842 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:08.595394 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e85830b4-be15-4f4c-ab66-62ecc464628f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:08.595842 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:08.595427 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-registry-tls\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:08.595842 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:08.595491 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 11:19:08.595842 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:08.595541 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e85830b4-be15-4f4c-ab66-62ecc464628f-insights-runtime-extractor-tls podName:e85830b4-be15-4f4c-ab66-62ecc464628f nodeName:}" failed. No retries permitted until 2026-07-14 11:19:16.595526211 +0000 UTC m=+50.756602682 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e85830b4-be15-4f4c-ab66-62ecc464628f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rc2bq" (UID: "e85830b4-be15-4f4c-ab66-62ecc464628f") : secret "insights-runtime-extractor-tls" not found Jul 14 11:19:08.595842 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:08.595609 2571 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 11:19:08.595842 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:08.595665 2571 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-559449c858-hbtxv: secret "image-registry-tls" not found Jul 14 11:19:08.595842 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:08.595727 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-registry-tls podName:584baebf-0808-4f20-adad-c8685a1eb082 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:16.595709711 +0000 UTC m=+50.756786170 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-registry-tls") pod "image-registry-559449c858-hbtxv" (UID: "584baebf-0808-4f20-adad-c8685a1eb082") : secret "image-registry-tls" not found Jul 14 11:19:08.596065 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:08.595910 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e85830b4-be15-4f4c-ab66-62ecc464628f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:08.696667 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:08.696623 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-metrics-tls\") pod \"dns-default-9lqh6\" (UID: \"6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5\") " pod="openshift-dns/dns-default-9lqh6" Jul 14 11:19:08.696792 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:08.696762 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 11:19:08.696832 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:08.696827 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-metrics-tls podName:6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:16.696810754 +0000 UTC m=+50.857887223 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-metrics-tls") pod "dns-default-9lqh6" (UID: "6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5") : secret "dns-default-metrics-tls" not found Jul 14 11:19:08.782202 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:08.782170 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:08.898134 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:08.898064 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/266bd207-c148-4d87-8eb7-baa4f02aa433-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-g6pxd\" (UID: \"266bd207-c148-4d87-8eb7-baa4f02aa433\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-g6pxd" Jul 14 11:19:08.898243 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:08.898176 2571 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 14 11:19:08.898243 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:08.898229 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/266bd207-c148-4d87-8eb7-baa4f02aa433-networking-console-plugin-cert podName:266bd207-c148-4d87-8eb7-baa4f02aa433 nodeName:}" failed. No retries permitted until 2026-07-14 11:19:16.898215878 +0000 UTC m=+51.059292323 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/266bd207-c148-4d87-8eb7-baa4f02aa433-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-g6pxd" (UID: "266bd207-c148-4d87-8eb7-baa4f02aa433") : secret "networking-console-plugin-cert" not found Jul 14 11:19:09.181681 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:09.181589 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:09.389245 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:09.389212 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:09.588379 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:09.588285 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:09.992739 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:09.992706 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:10.479934 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:10.479902 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:10.823754 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:10.823672 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:10.982085 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:10.982053 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:11.043602 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:11.043555 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:11.391688 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:11.391652 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:11.459499 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:11.459471 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:11.586018 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:11.585987 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:11.984259 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:11.984224 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:12.040972 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:12.040946 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:12.785958 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:12.785926 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:12.998344 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:12.998314 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:13.191159 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:13.191117 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:13.398804 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:13.398770 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:13.599578 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:13.599539 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:14.203770 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:14.203742 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:14.596898 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:14.596866 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:14.657876 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:14.657846 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:15.024232 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:15.024143 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:15.189082 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:15.189047 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:15.590185 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:15.590152 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:16.186181 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:16.186147 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:16.642981 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:16.642950 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:16.656966 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:16.656930 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-registry-tls\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:16.657078 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:16.657060 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e85830b4-be15-4f4c-ab66-62ecc464628f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:16.657196 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:16.657182 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 11:19:16.657257 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:16.657248 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e85830b4-be15-4f4c-ab66-62ecc464628f-insights-runtime-extractor-tls podName:e85830b4-be15-4f4c-ab66-62ecc464628f nodeName:}" failed. No retries permitted until 2026-07-14 11:19:32.657234657 +0000 UTC m=+66.818311102 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e85830b4-be15-4f4c-ab66-62ecc464628f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rc2bq" (UID: "e85830b4-be15-4f4c-ab66-62ecc464628f") : secret "insights-runtime-extractor-tls" not found Jul 14 11:19:16.660440 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:16.660421 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-registry-tls\") pod \"image-registry-559449c858-hbtxv\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:16.668066 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:16.668042 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:16.757593 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:16.757562 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-metrics-tls\") pod \"dns-default-9lqh6\" (UID: \"6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5\") " pod="openshift-dns/dns-default-9lqh6" Jul 14 11:19:16.760420 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:16.760390 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5-metrics-tls\") pod \"dns-default-9lqh6\" (UID: \"6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5\") " pod="openshift-dns/dns-default-9lqh6" Jul 14 11:19:16.769960 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:16.769934 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9lqh6" Jul 14 11:19:16.799267 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:16.799239 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:16.821029 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:16.820988 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-559449c858-hbtxv"] Jul 14 11:19:16.823930 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:19:16.823902 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod584baebf_0808_4f20_adad_c8685a1eb082.slice/crio-ff9b0eb055caba0abfe0b9f8f14fa6b125ca693f6fb9c1ed8bbbf0def8a1fb98 WatchSource:0}: Error finding container ff9b0eb055caba0abfe0b9f8f14fa6b125ca693f6fb9c1ed8bbbf0def8a1fb98: Status 404 returned error can't find the container with id ff9b0eb055caba0abfe0b9f8f14fa6b125ca693f6fb9c1ed8bbbf0def8a1fb98 Jul 14 11:19:16.917182 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:16.917157 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9lqh6"] Jul 14 11:19:16.919751 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:19:16.919727 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b1b3389_fa4a_4d72_b7f6_ffc78326fbc5.slice/crio-59e7373852c0275b7eefcace8dc0a3165134d004ecad4a5d4af0c9cd80087331 WatchSource:0}: Error finding container 59e7373852c0275b7eefcace8dc0a3165134d004ecad4a5d4af0c9cd80087331: Status 404 returned error can't find the container with id 59e7373852c0275b7eefcace8dc0a3165134d004ecad4a5d4af0c9cd80087331 Jul 14 11:19:16.959058 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:16.959034 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/266bd207-c148-4d87-8eb7-baa4f02aa433-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-g6pxd\" (UID: \"266bd207-c148-4d87-8eb7-baa4f02aa433\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-g6pxd" Jul 14 11:19:16.961073 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:16.961055 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/266bd207-c148-4d87-8eb7-baa4f02aa433-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-g6pxd\" (UID: \"266bd207-c148-4d87-8eb7-baa4f02aa433\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-g6pxd" Jul 14 11:19:16.996722 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:16.996691 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-g6pxd" Jul 14 11:19:16.997698 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:16.997676 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:17.129092 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:17.129059 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-g6pxd"] Jul 14 11:19:17.132118 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:19:17.132091 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod266bd207_c148_4d87_8eb7_baa4f02aa433.slice/crio-3de6cbae9fbc9e82a0dd9c7820e65e02c17147b5aacb7906faeb5ba4bfee6fc9 WatchSource:0}: Error finding container 3de6cbae9fbc9e82a0dd9c7820e65e02c17147b5aacb7906faeb5ba4bfee6fc9: Status 404 returned error can't find the container with id 3de6cbae9fbc9e82a0dd9c7820e65e02c17147b5aacb7906faeb5ba4bfee6fc9 Jul 14 11:19:17.598027 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:17.597990 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:17.614889 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:17.614842 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-g6pxd" event={"ID":"266bd207-c148-4d87-8eb7-baa4f02aa433","Type":"ContainerStarted","Data":"3de6cbae9fbc9e82a0dd9c7820e65e02c17147b5aacb7906faeb5ba4bfee6fc9"} Jul 14 11:19:17.616142 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:17.616102 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9lqh6" event={"ID":"6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5","Type":"ContainerStarted","Data":"59e7373852c0275b7eefcace8dc0a3165134d004ecad4a5d4af0c9cd80087331"} Jul 14 11:19:17.617669 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:17.617644 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-559449c858-hbtxv" event={"ID":"584baebf-0808-4f20-adad-c8685a1eb082","Type":"ContainerStarted","Data":"6aee7e75c393be24f1384019409bb6df7fb8f60f994a5301e4ec3166b3fcccdd"} Jul 14 11:19:17.617773 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:17.617686 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-559449c858-hbtxv" event={"ID":"584baebf-0808-4f20-adad-c8685a1eb082","Type":"ContainerStarted","Data":"ff9b0eb055caba0abfe0b9f8f14fa6b125ca693f6fb9c1ed8bbbf0def8a1fb98"} Jul 14 11:19:17.617773 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:17.617762 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:17.647526 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:17.647475 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-559449c858-hbtxv" podStartSLOduration=27.647457294 podStartE2EDuration="27.647457294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:18:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:17.646344111 +0000 UTC m=+51.807420577" watchObservedRunningTime="2026-07-14 11:19:17.647457294 +0000 UTC m=+51.808533849" Jul 14 11:19:17.795456 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:17.795419 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:19.628468 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:19.628430 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-g6pxd" event={"ID":"266bd207-c148-4d87-8eb7-baa4f02aa433","Type":"ContainerStarted","Data":"cee7aa30c6ebec893d2aefdfb7d9ff17dd0b3c73b50ec4963686d93c34f7de19"} Jul 14 11:19:19.630261 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:19.630234 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9lqh6" event={"ID":"6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5","Type":"ContainerStarted","Data":"c2fb0e5ec183d95f1eda5947356b4949eba69bb896eda26127ab38a19c237f3c"} Jul 14 11:19:19.630359 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:19.630284 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9lqh6" event={"ID":"6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5","Type":"ContainerStarted","Data":"f50756e2d9aed585bfc317e50de96fbf97307cf79fd9579e77499d3e20962504"} Jul 14 11:19:19.630359 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:19.630327 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9lqh6" Jul 14 11:19:19.684663 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:19.684608 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9lqh6" podStartSLOduration=17.361155378 podStartE2EDuration="19.684594549s" totalImagesPullingTime="2.323439171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:19:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:19.684588597 +0000 UTC m=+53.845665065" watchObservedRunningTime="2026-07-14 11:19:19.684594549 +0000 UTC m=+53.845671060" Jul 14 11:19:19.684788 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:19.684700 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-g6pxd" podStartSLOduration=16.579530937 podStartE2EDuration="18.684696443s" totalImagesPullingTime="2.105165506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:19.653191418 +0000 UTC m=+53.814267886" watchObservedRunningTime="2026-07-14 11:19:19.684696443 +0000 UTC m=+53.845772909" Jul 14 11:19:22.103341 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:22.103308 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-559449c858-hbtxv"] Jul 14 11:19:22.178991 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:22.178959 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:22.347077 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:22.347044 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:22.446582 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:22.446505 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:23.871643 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:23.871610 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:23.970786 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:23.970756 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:24.052647 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:24.052569 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:24.145792 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:24.145717 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:26.560923 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:26.560896 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5cglt" Jul 14 11:19:26.721500 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:26.721463 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:26.786051 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:26.786023 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:26.847598 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:26.847571 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:27.013550 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:27.013522 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:27.089150 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:27.089118 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:28.113491 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.113461 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:28.159556 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.159528 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-f55mq"] Jul 14 11:19:28.162547 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.162532 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-f55mq" Jul 14 11:19:28.166887 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.166863 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 14 11:19:28.167000 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.166872 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 14 11:19:28.167000 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.166863 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 14 11:19:28.167000 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.166874 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 14 11:19:28.167000 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.166865 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-lb5b9\"" Jul 14 11:19:28.167000 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.166920 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 14 11:19:28.175055 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.175037 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-f55mq"] Jul 14 11:19:28.201018 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.200993 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:28.240761 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.240734 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7221cd75-ecfe-4220-9716-a1ada0b79d76-metrics-client-ca\") pod \"prometheus-operator-64785b6675-f55mq\" (UID: \"7221cd75-ecfe-4220-9716-a1ada0b79d76\") " pod="openshift-monitoring/prometheus-operator-64785b6675-f55mq" Jul 14 11:19:28.240891 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.240774 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7221cd75-ecfe-4220-9716-a1ada0b79d76-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-f55mq\" (UID: \"7221cd75-ecfe-4220-9716-a1ada0b79d76\") " pod="openshift-monitoring/prometheus-operator-64785b6675-f55mq" Jul 14 11:19:28.240891 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.240797 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xpcj\" (UniqueName: \"kubernetes.io/projected/7221cd75-ecfe-4220-9716-a1ada0b79d76-kube-api-access-7xpcj\") pod \"prometheus-operator-64785b6675-f55mq\" (UID: \"7221cd75-ecfe-4220-9716-a1ada0b79d76\") " pod="openshift-monitoring/prometheus-operator-64785b6675-f55mq" Jul 14 11:19:28.240998 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.240934 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7221cd75-ecfe-4220-9716-a1ada0b79d76-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-f55mq\" (UID: \"7221cd75-ecfe-4220-9716-a1ada0b79d76\") " pod="openshift-monitoring/prometheus-operator-64785b6675-f55mq" Jul 14 11:19:28.311739 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.311703 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:28.341939 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.341909 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7221cd75-ecfe-4220-9716-a1ada0b79d76-metrics-client-ca\") pod \"prometheus-operator-64785b6675-f55mq\" (UID: \"7221cd75-ecfe-4220-9716-a1ada0b79d76\") " pod="openshift-monitoring/prometheus-operator-64785b6675-f55mq" Jul 14 11:19:28.342038 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.341952 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7221cd75-ecfe-4220-9716-a1ada0b79d76-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-f55mq\" (UID: \"7221cd75-ecfe-4220-9716-a1ada0b79d76\") " pod="openshift-monitoring/prometheus-operator-64785b6675-f55mq" Jul 14 11:19:28.342038 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.341976 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xpcj\" (UniqueName: \"kubernetes.io/projected/7221cd75-ecfe-4220-9716-a1ada0b79d76-kube-api-access-7xpcj\") pod \"prometheus-operator-64785b6675-f55mq\" (UID: \"7221cd75-ecfe-4220-9716-a1ada0b79d76\") " pod="openshift-monitoring/prometheus-operator-64785b6675-f55mq" Jul 14 11:19:28.342193 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.342169 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7221cd75-ecfe-4220-9716-a1ada0b79d76-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-f55mq\" (UID: \"7221cd75-ecfe-4220-9716-a1ada0b79d76\") " pod="openshift-monitoring/prometheus-operator-64785b6675-f55mq" Jul 14 11:19:28.342578 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.342560 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7221cd75-ecfe-4220-9716-a1ada0b79d76-metrics-client-ca\") pod \"prometheus-operator-64785b6675-f55mq\" (UID: \"7221cd75-ecfe-4220-9716-a1ada0b79d76\") " pod="openshift-monitoring/prometheus-operator-64785b6675-f55mq" Jul 14 11:19:28.344364 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.344341 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7221cd75-ecfe-4220-9716-a1ada0b79d76-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-f55mq\" (UID: \"7221cd75-ecfe-4220-9716-a1ada0b79d76\") " pod="openshift-monitoring/prometheus-operator-64785b6675-f55mq" Jul 14 11:19:28.344456 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.344434 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7221cd75-ecfe-4220-9716-a1ada0b79d76-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-f55mq\" (UID: \"7221cd75-ecfe-4220-9716-a1ada0b79d76\") " pod="openshift-monitoring/prometheus-operator-64785b6675-f55mq" Jul 14 11:19:28.352971 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.352949 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xpcj\" (UniqueName: \"kubernetes.io/projected/7221cd75-ecfe-4220-9716-a1ada0b79d76-kube-api-access-7xpcj\") pod \"prometheus-operator-64785b6675-f55mq\" (UID: \"7221cd75-ecfe-4220-9716-a1ada0b79d76\") " pod="openshift-monitoring/prometheus-operator-64785b6675-f55mq" Jul 14 11:19:28.470902 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.470818 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-f55mq" Jul 14 11:19:28.588020 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.587897 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-f55mq"] Jul 14 11:19:28.590313 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:19:28.590281 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7221cd75_ecfe_4220_9716_a1ada0b79d76.slice/crio-8d0c64035277afe0aca5b167c487ec0412dda0673eedf4251aebc15084371eb9 WatchSource:0}: Error finding container 8d0c64035277afe0aca5b167c487ec0412dda0673eedf4251aebc15084371eb9: Status 404 returned error can't find the container with id 8d0c64035277afe0aca5b167c487ec0412dda0673eedf4251aebc15084371eb9 Jul 14 11:19:28.662736 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:28.662703 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-f55mq" event={"ID":"7221cd75-ecfe-4220-9716-a1ada0b79d76","Type":"ContainerStarted","Data":"8d0c64035277afe0aca5b167c487ec0412dda0673eedf4251aebc15084371eb9"} Jul 14 11:19:29.637414 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:29.637380 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9lqh6" Jul 14 11:19:30.048929 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:30.048849 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:30.671327 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:30.671292 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-f55mq" event={"ID":"7221cd75-ecfe-4220-9716-a1ada0b79d76","Type":"ContainerStarted","Data":"6eb6ccd4b737b3e609b0a7aeedfcc19a7e9f1b59d4123bfbd5d86e09c534ed78"} Jul 14 11:19:30.671707 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:30.671342 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-f55mq" event={"ID":"7221cd75-ecfe-4220-9716-a1ada0b79d76","Type":"ContainerStarted","Data":"0b221b5d8efb7e87d46892db8377471dd3e36f47a356925564a0e9902addb9ef"} Jul 14 11:19:30.699855 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:30.699809 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-f55mq" podStartSLOduration=1.5271175750000001 podStartE2EDuration="2.699794608s" totalImagesPullingTime="1.172677033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:30.699729484 +0000 UTC m=+64.860805952" watchObservedRunningTime="2026-07-14 11:19:30.699794608 +0000 UTC m=+64.860871074" Jul 14 11:19:32.109219 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.109188 2571 patch_prober.go:28] interesting pod/image-registry-559449c858-hbtxv container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 14 11:19:32.109570 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.109246 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-559449c858-hbtxv" podUID="584baebf-0808-4f20-adad-c8685a1eb082" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 11:19:32.170084 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.170036 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs\") pod \"network-metrics-daemon-fwpln\" (UID: \"69592465-7d48-4eda-9662-361f7386ea86\") " pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:19:32.170084 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.170091 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6b2h\" (UniqueName: \"kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h\") pod \"network-check-target-9hcgm\" (UID: \"8d2c16f2-642c-4274-b5f2-3ee017c8b682\") " pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:19:32.170311 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.170120 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret\") pod \"global-pull-secret-syncer-882pd\" (UID: \"0fc0a645-d7c2-459a-8ecb-b0bae335b9ca\") " pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:19:32.173041 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.173017 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 11:19:32.174229 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.174209 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 11:19:32.174267 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.174241 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 11:19:32.182882 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.182861 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69592465-7d48-4eda-9662-361f7386ea86-metrics-certs\") pod \"network-metrics-daemon-fwpln\" (UID: \"69592465-7d48-4eda-9662-361f7386ea86\") " pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:19:32.182882 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.182871 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0fc0a645-d7c2-459a-8ecb-b0bae335b9ca-original-pull-secret\") pod \"global-pull-secret-syncer-882pd\" (UID: \"0fc0a645-d7c2-459a-8ecb-b0bae335b9ca\") " pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:19:32.183400 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.183379 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 11:19:32.192931 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.192907 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6b2h\" (UniqueName: \"kubernetes.io/projected/8d2c16f2-642c-4274-b5f2-3ee017c8b682-kube-api-access-n6b2h\") pod \"network-check-target-9hcgm\" (UID: \"8d2c16f2-642c-4274-b5f2-3ee017c8b682\") " pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:19:32.287276 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.287237 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:32.410572 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.410497 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q2rq4\"" Jul 14 11:19:32.416315 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.416299 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bm9fn\"" Jul 14 11:19:32.418404 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.418390 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:19:32.421295 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.421283 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-882pd" Jul 14 11:19:32.423845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.423833 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fwpln" Jul 14 11:19:32.500340 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.500262 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:32.570281 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.570253 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-9g65d"] Jul 14 11:19:32.571598 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:19:32.571574 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d2c16f2_642c_4274_b5f2_3ee017c8b682.slice/crio-f52ddf42310536fcd7bf3cedfe168a46b32149cf9fefc9eb7c207dba59b33bdb WatchSource:0}: Error finding container f52ddf42310536fcd7bf3cedfe168a46b32149cf9fefc9eb7c207dba59b33bdb: Status 404 returned error can't find the container with id f52ddf42310536fcd7bf3cedfe168a46b32149cf9fefc9eb7c207dba59b33bdb Jul 14 11:19:32.573305 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.573238 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" Jul 14 11:19:32.581205 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.581182 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9hcgm"] Jul 14 11:19:32.581950 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.581932 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-n48lw"] Jul 14 11:19:32.582079 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.582062 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 14 11:19:32.583690 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.583668 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-5dh4n\"" Jul 14 11:19:32.583845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.583696 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 14 11:19:32.583845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.583737 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 14 11:19:32.584042 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.584025 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.588911 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.588870 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5244g\"" Jul 14 11:19:32.589084 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.588945 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 14 11:19:32.589084 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.588970 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 14 11:19:32.589084 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.589080 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 14 11:19:32.590110 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.590088 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-9g65d"] Jul 14 11:19:32.592684 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.592618 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fwpln"] Jul 14 11:19:32.596102 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:19:32.596078 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69592465_7d48_4eda_9662_361f7386ea86.slice/crio-9d1c0cf77e366f8822b667b37225f70c2a65b90d765668b42a1c659aed33c8e7 WatchSource:0}: Error finding container 9d1c0cf77e366f8822b667b37225f70c2a65b90d765668b42a1c659aed33c8e7: Status 404 returned error can't find the container with id 9d1c0cf77e366f8822b667b37225f70c2a65b90d765668b42a1c659aed33c8e7 Jul 14 11:19:32.604430 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.604410 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-882pd"] Jul 14 11:19:32.607417 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:19:32.607384 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fc0a645_d7c2_459a_8ecb_b0bae335b9ca.slice/crio-895e8bdb1c04138f09871b503466f94b9caa5fd68d23180edb43a28eec98fd35 WatchSource:0}: Error finding container 895e8bdb1c04138f09871b503466f94b9caa5fd68d23180edb43a28eec98fd35: Status 404 returned error can't find the container with id 895e8bdb1c04138f09871b503466f94b9caa5fd68d23180edb43a28eec98fd35 Jul 14 11:19:32.612373 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.612353 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:32.673842 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.673764 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d6f473b7-1178-4f03-b497-7f507c0d54ff-node-exporter-tls\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.673842 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.673796 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckpwf\" (UniqueName: \"kubernetes.io/projected/d6f473b7-1178-4f03-b497-7f507c0d54ff-kube-api-access-ckpwf\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.673842 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.673815 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d6f473b7-1178-4f03-b497-7f507c0d54ff-sys\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.673842 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.673834 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4f499a28-6bfb-4628-9b11-3553c93a1a7b-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-9g65d\" (UID: \"4f499a28-6bfb-4628-9b11-3553c93a1a7b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" Jul 14 11:19:32.674060 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.673854 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f499a28-6bfb-4628-9b11-3553c93a1a7b-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-9g65d\" (UID: \"4f499a28-6bfb-4628-9b11-3553c93a1a7b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" Jul 14 11:19:32.674060 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.673935 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e85830b4-be15-4f4c-ab66-62ecc464628f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:32.674060 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.674014 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4f499a28-6bfb-4628-9b11-3553c93a1a7b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-9g65d\" (UID: \"4f499a28-6bfb-4628-9b11-3553c93a1a7b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" Jul 14 11:19:32.674060 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.674043 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d6f473b7-1178-4f03-b497-7f507c0d54ff-node-exporter-textfile\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.674060 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.674059 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d6f473b7-1178-4f03-b497-7f507c0d54ff-node-exporter-wtmp\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.674222 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.674078 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d6f473b7-1178-4f03-b497-7f507c0d54ff-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.674222 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.674129 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx6gv\" (UniqueName: \"kubernetes.io/projected/4f499a28-6bfb-4628-9b11-3553c93a1a7b-kube-api-access-xx6gv\") pod \"kube-state-metrics-785778ff74-9g65d\" (UID: \"4f499a28-6bfb-4628-9b11-3553c93a1a7b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" Jul 14 11:19:32.674222 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.674157 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d6f473b7-1178-4f03-b497-7f507c0d54ff-root\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.674222 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.674189 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6f473b7-1178-4f03-b497-7f507c0d54ff-metrics-client-ca\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.674222 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.674212 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d6f473b7-1178-4f03-b497-7f507c0d54ff-node-exporter-accelerators-collector-config\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.674405 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.674238 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f499a28-6bfb-4628-9b11-3553c93a1a7b-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-9g65d\" (UID: \"4f499a28-6bfb-4628-9b11-3553c93a1a7b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" Jul 14 11:19:32.674405 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.674261 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f499a28-6bfb-4628-9b11-3553c93a1a7b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-9g65d\" (UID: \"4f499a28-6bfb-4628-9b11-3553c93a1a7b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" Jul 14 11:19:32.676217 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.676197 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e85830b4-be15-4f4c-ab66-62ecc464628f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rc2bq\" (UID: \"e85830b4-be15-4f4c-ab66-62ecc464628f\") " pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:32.681178 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.681155 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-882pd" event={"ID":"0fc0a645-d7c2-459a-8ecb-b0bae335b9ca","Type":"ContainerStarted","Data":"895e8bdb1c04138f09871b503466f94b9caa5fd68d23180edb43a28eec98fd35"} Jul 14 11:19:32.682272 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.682251 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fwpln" event={"ID":"69592465-7d48-4eda-9662-361f7386ea86","Type":"ContainerStarted","Data":"9d1c0cf77e366f8822b667b37225f70c2a65b90d765668b42a1c659aed33c8e7"} Jul 14 11:19:32.683213 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.683194 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9hcgm" event={"ID":"8d2c16f2-642c-4274-b5f2-3ee017c8b682","Type":"ContainerStarted","Data":"f52ddf42310536fcd7bf3cedfe168a46b32149cf9fefc9eb7c207dba59b33bdb"} Jul 14 11:19:32.743185 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.743153 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:32.775416 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.775390 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d6f473b7-1178-4f03-b497-7f507c0d54ff-sys\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.775533 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.775424 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4f499a28-6bfb-4628-9b11-3553c93a1a7b-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-9g65d\" (UID: \"4f499a28-6bfb-4628-9b11-3553c93a1a7b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" Jul 14 11:19:32.775533 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.775443 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f499a28-6bfb-4628-9b11-3553c93a1a7b-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-9g65d\" (UID: \"4f499a28-6bfb-4628-9b11-3553c93a1a7b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" Jul 14 11:19:32.775533 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.775477 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4f499a28-6bfb-4628-9b11-3553c93a1a7b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-9g65d\" (UID: \"4f499a28-6bfb-4628-9b11-3553c93a1a7b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" Jul 14 11:19:32.775533 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.775491 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d6f473b7-1178-4f03-b497-7f507c0d54ff-sys\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.775533 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.775513 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d6f473b7-1178-4f03-b497-7f507c0d54ff-node-exporter-textfile\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.775796 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.775537 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d6f473b7-1178-4f03-b497-7f507c0d54ff-node-exporter-wtmp\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.775796 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.775564 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d6f473b7-1178-4f03-b497-7f507c0d54ff-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.775796 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.775712 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx6gv\" (UniqueName: \"kubernetes.io/projected/4f499a28-6bfb-4628-9b11-3553c93a1a7b-kube-api-access-xx6gv\") pod \"kube-state-metrics-785778ff74-9g65d\" (UID: \"4f499a28-6bfb-4628-9b11-3553c93a1a7b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" Jul 14 11:19:32.775796 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.775735 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d6f473b7-1178-4f03-b497-7f507c0d54ff-node-exporter-wtmp\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.775983 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.775796 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d6f473b7-1178-4f03-b497-7f507c0d54ff-root\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.775983 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.775829 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6f473b7-1178-4f03-b497-7f507c0d54ff-metrics-client-ca\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.775983 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.775854 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d6f473b7-1178-4f03-b497-7f507c0d54ff-node-exporter-accelerators-collector-config\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.775983 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.775880 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f499a28-6bfb-4628-9b11-3553c93a1a7b-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-9g65d\" (UID: \"4f499a28-6bfb-4628-9b11-3553c93a1a7b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" Jul 14 11:19:32.775983 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.775906 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f499a28-6bfb-4628-9b11-3553c93a1a7b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-9g65d\" (UID: \"4f499a28-6bfb-4628-9b11-3553c93a1a7b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" Jul 14 11:19:32.775983 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.775908 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4f499a28-6bfb-4628-9b11-3553c93a1a7b-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-9g65d\" (UID: \"4f499a28-6bfb-4628-9b11-3553c93a1a7b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" Jul 14 11:19:32.775983 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.775940 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d6f473b7-1178-4f03-b497-7f507c0d54ff-node-exporter-tls\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.775983 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.775967 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckpwf\" (UniqueName: \"kubernetes.io/projected/d6f473b7-1178-4f03-b497-7f507c0d54ff-kube-api-access-ckpwf\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.776361 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.775991 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d6f473b7-1178-4f03-b497-7f507c0d54ff-root\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.776361 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.775939 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d6f473b7-1178-4f03-b497-7f507c0d54ff-node-exporter-textfile\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.776361 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:32.776174 2571 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 14 11:19:32.776361 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:32.776266 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f499a28-6bfb-4628-9b11-3553c93a1a7b-kube-state-metrics-tls podName:4f499a28-6bfb-4628-9b11-3553c93a1a7b nodeName:}" failed. No retries permitted until 2026-07-14 11:19:33.2762462 +0000 UTC m=+67.437322657 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/4f499a28-6bfb-4628-9b11-3553c93a1a7b-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-9g65d" (UID: "4f499a28-6bfb-4628-9b11-3553c93a1a7b") : secret "kube-state-metrics-tls" not found Jul 14 11:19:32.776361 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.776299 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f499a28-6bfb-4628-9b11-3553c93a1a7b-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-9g65d\" (UID: \"4f499a28-6bfb-4628-9b11-3553c93a1a7b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" Jul 14 11:19:32.776606 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.776371 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4f499a28-6bfb-4628-9b11-3553c93a1a7b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-9g65d\" (UID: \"4f499a28-6bfb-4628-9b11-3553c93a1a7b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" Jul 14 11:19:32.776606 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.776553 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d6f473b7-1178-4f03-b497-7f507c0d54ff-metrics-client-ca\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.776798 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.776774 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d6f473b7-1178-4f03-b497-7f507c0d54ff-node-exporter-accelerators-collector-config\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.778219 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.778198 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d6f473b7-1178-4f03-b497-7f507c0d54ff-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.778344 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.778327 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4f499a28-6bfb-4628-9b11-3553c93a1a7b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-9g65d\" (UID: \"4f499a28-6bfb-4628-9b11-3553c93a1a7b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" Jul 14 11:19:32.778344 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.778337 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d6f473b7-1178-4f03-b497-7f507c0d54ff-node-exporter-tls\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.786506 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.786489 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckpwf\" (UniqueName: \"kubernetes.io/projected/d6f473b7-1178-4f03-b497-7f507c0d54ff-kube-api-access-ckpwf\") pod \"node-exporter-n48lw\" (UID: \"d6f473b7-1178-4f03-b497-7f507c0d54ff\") " pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.786740 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.786724 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx6gv\" (UniqueName: \"kubernetes.io/projected/4f499a28-6bfb-4628-9b11-3553c93a1a7b-kube-api-access-xx6gv\") pod \"kube-state-metrics-785778ff74-9g65d\" (UID: \"4f499a28-6bfb-4628-9b11-3553c93a1a7b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" Jul 14 11:19:32.880909 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.880871 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xrcwt\"" Jul 14 11:19:32.888461 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.888438 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rc2bq" Jul 14 11:19:32.895114 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.895094 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n48lw" Jul 14 11:19:32.905875 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:19:32.905851 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6f473b7_1178_4f03_b497_7f507c0d54ff.slice/crio-eb77b7805905baf80656a47c486c940fc163e3debc347871a8d0ec9128311dde WatchSource:0}: Error finding container eb77b7805905baf80656a47c486c940fc163e3debc347871a8d0ec9128311dde: Status 404 returned error can't find the container with id eb77b7805905baf80656a47c486c940fc163e3debc347871a8d0ec9128311dde Jul 14 11:19:32.924942 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:32.924836 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:33.020808 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:33.020714 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:33.037527 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:33.037494 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rc2bq"] Jul 14 11:19:33.043226 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:19:33.043193 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode85830b4_be15_4f4c_ab66_62ecc464628f.slice/crio-aa6437142a3aa9625b6b5c2c5e48beba627eea90869f07e191082f3521b7e918 WatchSource:0}: Error finding container aa6437142a3aa9625b6b5c2c5e48beba627eea90869f07e191082f3521b7e918: Status 404 returned error can't find the container with id aa6437142a3aa9625b6b5c2c5e48beba627eea90869f07e191082f3521b7e918 Jul 14 11:19:33.121372 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:33.121306 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:33.280369 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:33.280285 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f499a28-6bfb-4628-9b11-3553c93a1a7b-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-9g65d\" (UID: \"4f499a28-6bfb-4628-9b11-3553c93a1a7b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" Jul 14 11:19:33.284601 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:33.284535 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4f499a28-6bfb-4628-9b11-3553c93a1a7b-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-9g65d\" (UID: \"4f499a28-6bfb-4628-9b11-3553c93a1a7b\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" Jul 14 11:19:33.483990 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:33.483739 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" Jul 14 11:19:33.548839 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:33.548743 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:33.646391 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:33.646354 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:33.692929 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:33.692846 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n48lw" event={"ID":"d6f473b7-1178-4f03-b497-7f507c0d54ff","Type":"ContainerStarted","Data":"eb77b7805905baf80656a47c486c940fc163e3debc347871a8d0ec9128311dde"} Jul 14 11:19:33.694758 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:33.694722 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rc2bq" event={"ID":"e85830b4-be15-4f4c-ab66-62ecc464628f","Type":"ContainerStarted","Data":"aa6437142a3aa9625b6b5c2c5e48beba627eea90869f07e191082f3521b7e918"} Jul 14 11:19:33.974036 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:33.973993 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:34.067979 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:34.067894 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:34.223801 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:34.223712 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-9g65d"] Jul 14 11:19:34.226706 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:19:34.226679 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f499a28_6bfb_4628_9b11_3553c93a1a7b.slice/crio-155dafce23d5457b5ae1639fa05bc65ad5520fae926c14ce6ef5ff0cb413a3b1 WatchSource:0}: Error finding container 155dafce23d5457b5ae1639fa05bc65ad5520fae926c14ce6ef5ff0cb413a3b1: Status 404 returned error can't find the container with id 155dafce23d5457b5ae1639fa05bc65ad5520fae926c14ce6ef5ff0cb413a3b1 Jul 14 11:19:34.694981 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:34.694891 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:34.703115 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:34.703038 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" event={"ID":"4f499a28-6bfb-4628-9b11-3553c93a1a7b","Type":"ContainerStarted","Data":"155dafce23d5457b5ae1639fa05bc65ad5520fae926c14ce6ef5ff0cb413a3b1"} Jul 14 11:19:34.707491 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:34.707449 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fwpln" event={"ID":"69592465-7d48-4eda-9662-361f7386ea86","Type":"ContainerStarted","Data":"ff001be6eca856d0d03c39dc8bb989dfe019f2ed90f5aebad4e598339534917c"} Jul 14 11:19:34.707684 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:34.707509 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fwpln" event={"ID":"69592465-7d48-4eda-9662-361f7386ea86","Type":"ContainerStarted","Data":"82c1e9dda043429d972ba66dc820f38c10dbea70a7779d6dfbc00b73458a88cc"} Jul 14 11:19:34.710465 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:34.710434 2571 generic.go:356] "Generic (PLEG): container finished" podID="d6f473b7-1178-4f03-b497-7f507c0d54ff" containerID="3dd086aa01d6960fca9f20ce9dd5018b811c8e4f26dd85749dfa1b879d36ea6b" exitCode=0 Jul 14 11:19:34.710567 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:34.710482 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n48lw" event={"ID":"d6f473b7-1178-4f03-b497-7f507c0d54ff","Type":"ContainerDied","Data":"3dd086aa01d6960fca9f20ce9dd5018b811c8e4f26dd85749dfa1b879d36ea6b"} Jul 14 11:19:34.728668 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:34.728602 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fwpln" podStartSLOduration=67.280223043 podStartE2EDuration="1m8.728583067s" totalImagesPullingTime="1.448360024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:34.726919651 +0000 UTC m=+68.887996119" watchObservedRunningTime="2026-07-14 11:19:34.728583067 +0000 UTC m=+68.889659538" Jul 14 11:19:34.840464 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:34.840438 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:34.939468 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:34.939405 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:35.558449 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:35.558405 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:35.642548 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:35.642435 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:35.719506 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:35.719470 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n48lw" event={"ID":"d6f473b7-1178-4f03-b497-7f507c0d54ff","Type":"ContainerStarted","Data":"872533088dea9ff067038942869a98c53028b6bb99666f1edbdcaee048ffc6ef"} Jul 14 11:19:35.719688 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:35.719530 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n48lw" event={"ID":"d6f473b7-1178-4f03-b497-7f507c0d54ff","Type":"ContainerStarted","Data":"63a8b0b61afc71168711ac2cbe746bfdfa8735e41e95f2039b89186c25a77d91"} Jul 14 11:19:35.746670 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:35.746478 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-n48lw" podStartSLOduration=2.46396619 podStartE2EDuration="3.746456971s" totalImagesPullingTime="1.143774469s" totalInitContainerRuntime="138.716312ms" isStatefulPod=true podCreationTimestamp="2026-07-14 11:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:35.742486942 +0000 UTC m=+69.903563410" watchObservedRunningTime="2026-07-14 11:19:35.746456971 +0000 UTC m=+69.907533440" Jul 14 11:19:35.801237 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:35.801203 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-9h6fb"] Jul 14 11:19:35.814253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:35.814180 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-9h6fb" Jul 14 11:19:35.814754 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:35.814729 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-9h6fb"] Jul 14 11:19:35.817179 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:35.817154 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-2jpfm\"" Jul 14 11:19:35.817675 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:35.817655 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 14 11:19:35.817789 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:35.817673 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 14 11:19:35.820767 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:35.820726 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:35.904114 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:35.904080 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkznn\" (UniqueName: \"kubernetes.io/projected/0fe38157-e3ad-4fdb-a833-8418983f208f-kube-api-access-lkznn\") pod \"downloads-68c9cbb497-9h6fb\" (UID: \"0fe38157-e3ad-4fdb-a833-8418983f208f\") " pod="openshift-console/downloads-68c9cbb497-9h6fb" Jul 14 11:19:35.917842 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:35.917813 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:36.005285 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:36.005232 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkznn\" (UniqueName: \"kubernetes.io/projected/0fe38157-e3ad-4fdb-a833-8418983f208f-kube-api-access-lkznn\") pod \"downloads-68c9cbb497-9h6fb\" (UID: \"0fe38157-e3ad-4fdb-a833-8418983f208f\") " pod="openshift-console/downloads-68c9cbb497-9h6fb" Jul 14 11:19:36.015059 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:36.015015 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:36.016733 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:36.016695 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkznn\" (UniqueName: \"kubernetes.io/projected/0fe38157-e3ad-4fdb-a833-8418983f208f-kube-api-access-lkznn\") pod \"downloads-68c9cbb497-9h6fb\" (UID: \"0fe38157-e3ad-4fdb-a833-8418983f208f\") " pod="openshift-console/downloads-68c9cbb497-9h6fb" Jul 14 11:19:36.116336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:36.116265 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:36.126269 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:36.126246 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-9h6fb" Jul 14 11:19:36.491863 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:36.491783 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:36.595431 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:36.595389 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:36.870406 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:36.870370 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:36.984702 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:36.984662 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6f589fd54-c6cfb"] Jul 14 11:19:36.992666 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:36.992621 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:37.006171 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.006148 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6f589fd54-c6cfb"] Jul 14 11:19:37.006295 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.006245 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.009397 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.009370 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 14 11:19:37.010861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.010837 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 14 11:19:37.010994 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.010917 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-2rvg6\"" Jul 14 11:19:37.010994 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.010950 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 14 11:19:37.011236 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.011214 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 14 11:19:37.011349 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.011252 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2f8b3pmvgbjgb\"" Jul 14 11:19:37.114555 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.114521 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1edfd667-50c9-4c39-9924-6770649f9c4a-secret-metrics-server-tls\") pod \"metrics-server-6f589fd54-c6cfb\" (UID: \"1edfd667-50c9-4c39-9924-6770649f9c4a\") " pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.114555 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.114552 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1edfd667-50c9-4c39-9924-6770649f9c4a-client-ca-bundle\") pod \"metrics-server-6f589fd54-c6cfb\" (UID: \"1edfd667-50c9-4c39-9924-6770649f9c4a\") " pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.114745 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.114583 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jmlx\" (UniqueName: \"kubernetes.io/projected/1edfd667-50c9-4c39-9924-6770649f9c4a-kube-api-access-8jmlx\") pod \"metrics-server-6f589fd54-c6cfb\" (UID: \"1edfd667-50c9-4c39-9924-6770649f9c4a\") " pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.114745 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.114678 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1edfd667-50c9-4c39-9924-6770649f9c4a-secret-metrics-server-client-certs\") pod \"metrics-server-6f589fd54-c6cfb\" (UID: \"1edfd667-50c9-4c39-9924-6770649f9c4a\") " pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.114745 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.114710 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1edfd667-50c9-4c39-9924-6770649f9c4a-audit-log\") pod \"metrics-server-6f589fd54-c6cfb\" (UID: \"1edfd667-50c9-4c39-9924-6770649f9c4a\") " pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.114745 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.114729 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1edfd667-50c9-4c39-9924-6770649f9c4a-metrics-server-audit-profiles\") pod \"metrics-server-6f589fd54-c6cfb\" (UID: \"1edfd667-50c9-4c39-9924-6770649f9c4a\") " pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.114865 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.114749 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1edfd667-50c9-4c39-9924-6770649f9c4a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f589fd54-c6cfb\" (UID: \"1edfd667-50c9-4c39-9924-6770649f9c4a\") " pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.215736 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.215657 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1edfd667-50c9-4c39-9924-6770649f9c4a-secret-metrics-server-client-certs\") pod \"metrics-server-6f589fd54-c6cfb\" (UID: \"1edfd667-50c9-4c39-9924-6770649f9c4a\") " pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.215736 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.215700 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1edfd667-50c9-4c39-9924-6770649f9c4a-audit-log\") pod \"metrics-server-6f589fd54-c6cfb\" (UID: \"1edfd667-50c9-4c39-9924-6770649f9c4a\") " pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.215736 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.215725 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1edfd667-50c9-4c39-9924-6770649f9c4a-metrics-server-audit-profiles\") pod \"metrics-server-6f589fd54-c6cfb\" (UID: \"1edfd667-50c9-4c39-9924-6770649f9c4a\") " pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.215986 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.215745 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1edfd667-50c9-4c39-9924-6770649f9c4a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f589fd54-c6cfb\" (UID: \"1edfd667-50c9-4c39-9924-6770649f9c4a\") " pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.215986 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.215780 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1edfd667-50c9-4c39-9924-6770649f9c4a-secret-metrics-server-tls\") pod \"metrics-server-6f589fd54-c6cfb\" (UID: \"1edfd667-50c9-4c39-9924-6770649f9c4a\") " pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.215986 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.215797 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1edfd667-50c9-4c39-9924-6770649f9c4a-client-ca-bundle\") pod \"metrics-server-6f589fd54-c6cfb\" (UID: \"1edfd667-50c9-4c39-9924-6770649f9c4a\") " pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.215986 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.215837 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jmlx\" (UniqueName: \"kubernetes.io/projected/1edfd667-50c9-4c39-9924-6770649f9c4a-kube-api-access-8jmlx\") pod \"metrics-server-6f589fd54-c6cfb\" (UID: \"1edfd667-50c9-4c39-9924-6770649f9c4a\") " pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.216396 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.216348 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1edfd667-50c9-4c39-9924-6770649f9c4a-audit-log\") pod \"metrics-server-6f589fd54-c6cfb\" (UID: \"1edfd667-50c9-4c39-9924-6770649f9c4a\") " pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.216623 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.216603 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1edfd667-50c9-4c39-9924-6770649f9c4a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6f589fd54-c6cfb\" (UID: \"1edfd667-50c9-4c39-9924-6770649f9c4a\") " pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.216935 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.216909 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1edfd667-50c9-4c39-9924-6770649f9c4a-metrics-server-audit-profiles\") pod \"metrics-server-6f589fd54-c6cfb\" (UID: \"1edfd667-50c9-4c39-9924-6770649f9c4a\") " pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.218613 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.218594 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1edfd667-50c9-4c39-9924-6770649f9c4a-client-ca-bundle\") pod \"metrics-server-6f589fd54-c6cfb\" (UID: \"1edfd667-50c9-4c39-9924-6770649f9c4a\") " pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.218785 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.218753 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1edfd667-50c9-4c39-9924-6770649f9c4a-secret-metrics-server-client-certs\") pod \"metrics-server-6f589fd54-c6cfb\" (UID: \"1edfd667-50c9-4c39-9924-6770649f9c4a\") " pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.219096 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.219074 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1edfd667-50c9-4c39-9924-6770649f9c4a-secret-metrics-server-tls\") pod \"metrics-server-6f589fd54-c6cfb\" (UID: \"1edfd667-50c9-4c39-9924-6770649f9c4a\") " pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.224411 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.224387 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jmlx\" (UniqueName: \"kubernetes.io/projected/1edfd667-50c9-4c39-9924-6770649f9c4a-kube-api-access-8jmlx\") pod \"metrics-server-6f589fd54-c6cfb\" (UID: \"1edfd667-50c9-4c39-9924-6770649f9c4a\") " pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.283305 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.283265 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:37.316326 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.316224 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:37.359098 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.359064 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:37.486236 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.486159 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:37.806159 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.806067 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:37.907310 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:37.907277 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:38.743718 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:38.743672 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:38.943755 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:38.943717 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:39.064749 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:39.064654 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:39.330187 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:39.330094 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:39.432347 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:39.432317 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:40.750232 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:40.750204 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6f589fd54-c6cfb"] Jul 14 11:19:40.970422 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:19:40.970396 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1edfd667_50c9_4c39_9924_6770649f9c4a.slice/crio-8f10101364f9410dc94f8ce35201e014e4202c4c195af828f071f76724852095 WatchSource:0}: Error finding container 8f10101364f9410dc94f8ce35201e014e4202c4c195af828f071f76724852095: Status 404 returned error can't find the container with id 8f10101364f9410dc94f8ce35201e014e4202c4c195af828f071f76724852095 Jul 14 11:19:41.127094 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:41.127069 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-9h6fb"] Jul 14 11:19:41.129943 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:19:41.129889 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fe38157_e3ad_4fdb_a833_8418983f208f.slice/crio-3efb85d09288e31311fd6fe18481e94187ee8157a836971cba51c6d82cc5c1c3 WatchSource:0}: Error finding container 3efb85d09288e31311fd6fe18481e94187ee8157a836971cba51c6d82cc5c1c3: Status 404 returned error can't find the container with id 3efb85d09288e31311fd6fe18481e94187ee8157a836971cba51c6d82cc5c1c3 Jul 14 11:19:41.743682 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:41.743646 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" event={"ID":"4f499a28-6bfb-4628-9b11-3553c93a1a7b","Type":"ContainerStarted","Data":"28add77db7d984822f27d5a5f56c69299051ea31f4b5c4d7d4944d9e116e9085"} Jul 14 11:19:41.743866 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:41.743717 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" event={"ID":"4f499a28-6bfb-4628-9b11-3553c93a1a7b","Type":"ContainerStarted","Data":"0fbcb351bdb37c9b973b229d08afcd156a14251d9dea83f514deda7b220f6407"} Jul 14 11:19:41.743866 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:41.743752 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" event={"ID":"4f499a28-6bfb-4628-9b11-3553c93a1a7b","Type":"ContainerStarted","Data":"e5a7b49976d078887669ff0a3871d8dd437646874abbe3ff343f04e8b738d639"} Jul 14 11:19:41.745569 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:41.745404 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-882pd" event={"ID":"0fc0a645-d7c2-459a-8ecb-b0bae335b9ca","Type":"ContainerStarted","Data":"76bf3ada3dedb49194e7d7cc2db7435a88ec1944a76fcff185ba6fdb34ab9c33"} Jul 14 11:19:41.747049 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:41.747023 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9hcgm" event={"ID":"8d2c16f2-642c-4274-b5f2-3ee017c8b682","Type":"ContainerStarted","Data":"3623ee04aaccc6e688b290d373bc8b9543575ff4ebb3179b7d1e3ad12830f1c3"} Jul 14 11:19:41.747225 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:41.747080 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:19:41.748816 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:41.748786 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rc2bq" event={"ID":"e85830b4-be15-4f4c-ab66-62ecc464628f","Type":"ContainerStarted","Data":"d37925742ee289f5530240b65f2e9086cb5940b03bf6a01ee58f0db42efafcaf"} Jul 14 11:19:41.750277 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:41.750253 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-9h6fb" event={"ID":"0fe38157-e3ad-4fdb-a833-8418983f208f","Type":"ContainerStarted","Data":"3efb85d09288e31311fd6fe18481e94187ee8157a836971cba51c6d82cc5c1c3"} Jul 14 11:19:41.751571 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:41.751551 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" event={"ID":"1edfd667-50c9-4c39-9924-6770649f9c4a","Type":"ContainerStarted","Data":"8f10101364f9410dc94f8ce35201e014e4202c4c195af828f071f76724852095"} Jul 14 11:19:41.765535 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:41.765486 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-9g65d" podStartSLOduration=3.383604439 podStartE2EDuration="9.765471832s" totalImagesPullingTime="6.381867393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:41.763403773 +0000 UTC m=+75.924480246" watchObservedRunningTime="2026-07-14 11:19:41.765471832 +0000 UTC m=+75.926548298" Jul 14 11:19:41.779313 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:41.779266 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-882pd" podStartSLOduration=67.41897657 podStartE2EDuration="1m15.77924575s" totalImagesPullingTime="8.36026918s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:41.778695786 +0000 UTC m=+75.939772258" watchObservedRunningTime="2026-07-14 11:19:41.77924575 +0000 UTC m=+75.940322214" Jul 14 11:19:41.795884 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:41.795831 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9hcgm" podStartSLOduration=67.397182558 podStartE2EDuration="1m15.795812806s" totalImagesPullingTime="8.398630248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:41.795373863 +0000 UTC m=+75.956450328" watchObservedRunningTime="2026-07-14 11:19:41.795812806 +0000 UTC m=+75.956889276" Jul 14 11:19:42.010328 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:42.010192 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:42.082955 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:42.082916 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:42.108985 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:42.108956 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:42.760855 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:42.760755 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rc2bq" event={"ID":"e85830b4-be15-4f4c-ab66-62ecc464628f","Type":"ContainerStarted","Data":"1a690fe83ec564ef199ace14854530b91a512d66e15dac39c33a4091faef5961"} Jul 14 11:19:42.765183 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:42.764334 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" event={"ID":"1edfd667-50c9-4c39-9924-6770649f9c4a","Type":"ContainerStarted","Data":"140e026a75058fc94968ea90764dacaf31f270a73512832e7e8d2eadcb3ec26b"} Jul 14 11:19:42.803883 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:42.803490 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" podStartSLOduration=5.45320031 podStartE2EDuration="6.803472831s" totalImagesPullingTime="1.350272521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:19:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:42.802093272 +0000 UTC m=+76.963169740" watchObservedRunningTime="2026-07-14 11:19:42.803472831 +0000 UTC m=+76.964549275" Jul 14 11:19:44.238328 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:44.238289 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:44.329702 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:44.329649 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:45.359196 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:45.359160 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:45.422375 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:45.422342 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:45.778277 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:45.778193 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rc2bq" event={"ID":"e85830b4-be15-4f4c-ab66-62ecc464628f","Type":"ContainerStarted","Data":"cd59f7bf4f1e5a8895340cf510269f393e043fe5585140c81b838c680157cab7"} Jul 14 11:19:45.797769 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:45.797707 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rc2bq" podStartSLOduration=34.068478815 podStartE2EDuration="45.79768567s" totalImagesPullingTime="11.729206855s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 11:19:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:45.797489856 +0000 UTC m=+79.958566348" watchObservedRunningTime="2026-07-14 11:19:45.79768567 +0000 UTC m=+79.958762138" Jul 14 11:19:46.706979 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.706942 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6454d4ff44-7cxct"] Jul 14 11:19:46.709379 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.709357 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:46.713781 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.713557 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 14 11:19:46.713781 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.713557 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 14 11:19:46.713781 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.713675 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 14 11:19:46.713781 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.713680 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hjsq7\"" Jul 14 11:19:46.713781 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.713700 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 14 11:19:46.714080 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.713788 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 14 11:19:46.722344 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.722324 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6454d4ff44-7cxct"] Jul 14 11:19:46.807879 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.807854 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4af9286d-a261-4209-8290-59e9ba383bfb-oauth-serving-cert\") pod \"console-6454d4ff44-7cxct\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:46.808017 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.807893 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4af9286d-a261-4209-8290-59e9ba383bfb-service-ca\") pod \"console-6454d4ff44-7cxct\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:46.808017 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.807919 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4af9286d-a261-4209-8290-59e9ba383bfb-console-config\") pod \"console-6454d4ff44-7cxct\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:46.808017 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.807994 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4af9286d-a261-4209-8290-59e9ba383bfb-console-serving-cert\") pod \"console-6454d4ff44-7cxct\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:46.808122 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.808049 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4af9286d-a261-4209-8290-59e9ba383bfb-console-oauth-config\") pod \"console-6454d4ff44-7cxct\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:46.808122 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.808066 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnmlt\" (UniqueName: \"kubernetes.io/projected/4af9286d-a261-4209-8290-59e9ba383bfb-kube-api-access-qnmlt\") pod \"console-6454d4ff44-7cxct\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:46.895800 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.895767 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:46.909353 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.909327 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4af9286d-a261-4209-8290-59e9ba383bfb-oauth-serving-cert\") pod \"console-6454d4ff44-7cxct\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:46.909480 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.909393 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4af9286d-a261-4209-8290-59e9ba383bfb-service-ca\") pod \"console-6454d4ff44-7cxct\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:46.909480 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.909422 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4af9286d-a261-4209-8290-59e9ba383bfb-console-config\") pod \"console-6454d4ff44-7cxct\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:46.909584 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.909483 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4af9286d-a261-4209-8290-59e9ba383bfb-console-serving-cert\") pod \"console-6454d4ff44-7cxct\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:46.909584 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.909533 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4af9286d-a261-4209-8290-59e9ba383bfb-console-oauth-config\") pod \"console-6454d4ff44-7cxct\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:46.909584 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.909555 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnmlt\" (UniqueName: \"kubernetes.io/projected/4af9286d-a261-4209-8290-59e9ba383bfb-kube-api-access-qnmlt\") pod \"console-6454d4ff44-7cxct\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:46.910198 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.910144 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4af9286d-a261-4209-8290-59e9ba383bfb-oauth-serving-cert\") pod \"console-6454d4ff44-7cxct\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:46.910468 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.910446 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4af9286d-a261-4209-8290-59e9ba383bfb-console-config\") pod \"console-6454d4ff44-7cxct\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:46.910468 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.910459 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4af9286d-a261-4209-8290-59e9ba383bfb-service-ca\") pod \"console-6454d4ff44-7cxct\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:46.912215 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.912194 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4af9286d-a261-4209-8290-59e9ba383bfb-console-oauth-config\") pod \"console-6454d4ff44-7cxct\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:46.912377 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.912361 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4af9286d-a261-4209-8290-59e9ba383bfb-console-serving-cert\") pod \"console-6454d4ff44-7cxct\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:46.922269 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.922241 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnmlt\" (UniqueName: \"kubernetes.io/projected/4af9286d-a261-4209-8290-59e9ba383bfb-kube-api-access-qnmlt\") pod \"console-6454d4ff44-7cxct\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:46.996955 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:46.996878 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:47.019936 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.019913 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:47.122680 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.122619 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-559449c858-hbtxv" podUID="584baebf-0808-4f20-adad-c8685a1eb082" containerName="registry" containerID="cri-o://6aee7e75c393be24f1384019409bb6df7fb8f60f994a5301e4ec3166b3fcccdd" gracePeriod=30 Jul 14 11:19:47.165685 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.165658 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6454d4ff44-7cxct"] Jul 14 11:19:47.169548 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:19:47.169522 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4af9286d_a261_4209_8290_59e9ba383bfb.slice/crio-1a91116a15f09a7c3dc07c9f201368700a435c6d825585caab7e478eb1876378 WatchSource:0}: Error finding container 1a91116a15f09a7c3dc07c9f201368700a435c6d825585caab7e478eb1876378: Status 404 returned error can't find the container with id 1a91116a15f09a7c3dc07c9f201368700a435c6d825585caab7e478eb1876378 Jul 14 11:19:47.360385 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.360362 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:47.514477 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.514444 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/584baebf-0808-4f20-adad-c8685a1eb082-ca-trust-extracted\") pod \"584baebf-0808-4f20-adad-c8685a1eb082\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " Jul 14 11:19:47.514653 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.514494 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-registry-tls\") pod \"584baebf-0808-4f20-adad-c8685a1eb082\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " Jul 14 11:19:47.514653 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.514539 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kr97\" (UniqueName: \"kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-kube-api-access-4kr97\") pod \"584baebf-0808-4f20-adad-c8685a1eb082\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " Jul 14 11:19:47.514653 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.514580 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-bound-sa-token\") pod \"584baebf-0808-4f20-adad-c8685a1eb082\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " Jul 14 11:19:47.514653 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.514613 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/584baebf-0808-4f20-adad-c8685a1eb082-image-registry-private-configuration\") pod \"584baebf-0808-4f20-adad-c8685a1eb082\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " Jul 14 11:19:47.514861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.514654 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/584baebf-0808-4f20-adad-c8685a1eb082-registry-certificates\") pod \"584baebf-0808-4f20-adad-c8685a1eb082\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " Jul 14 11:19:47.514861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.514698 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/584baebf-0808-4f20-adad-c8685a1eb082-trusted-ca\") pod \"584baebf-0808-4f20-adad-c8685a1eb082\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " Jul 14 11:19:47.514861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.514723 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/584baebf-0808-4f20-adad-c8685a1eb082-installation-pull-secrets\") pod \"584baebf-0808-4f20-adad-c8685a1eb082\" (UID: \"584baebf-0808-4f20-adad-c8685a1eb082\") " Jul 14 11:19:47.515339 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.515195 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/584baebf-0808-4f20-adad-c8685a1eb082-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "584baebf-0808-4f20-adad-c8685a1eb082" (UID: "584baebf-0808-4f20-adad-c8685a1eb082"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:19:47.515670 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.515600 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/584baebf-0808-4f20-adad-c8685a1eb082-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "584baebf-0808-4f20-adad-c8685a1eb082" (UID: "584baebf-0808-4f20-adad-c8685a1eb082"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:19:47.517548 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.517521 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "584baebf-0808-4f20-adad-c8685a1eb082" (UID: "584baebf-0808-4f20-adad-c8685a1eb082"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:19:47.517663 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.517621 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/584baebf-0808-4f20-adad-c8685a1eb082-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "584baebf-0808-4f20-adad-c8685a1eb082" (UID: "584baebf-0808-4f20-adad-c8685a1eb082"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:19:47.517739 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.517693 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "584baebf-0808-4f20-adad-c8685a1eb082" (UID: "584baebf-0808-4f20-adad-c8685a1eb082"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:19:47.517797 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.517748 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-kube-api-access-4kr97" (OuterVolumeSpecName: "kube-api-access-4kr97") pod "584baebf-0808-4f20-adad-c8685a1eb082" (UID: "584baebf-0808-4f20-adad-c8685a1eb082"). InnerVolumeSpecName "kube-api-access-4kr97". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:19:47.518013 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.517990 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/584baebf-0808-4f20-adad-c8685a1eb082-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "584baebf-0808-4f20-adad-c8685a1eb082" (UID: "584baebf-0808-4f20-adad-c8685a1eb082"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:19:47.525929 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.525905 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/584baebf-0808-4f20-adad-c8685a1eb082-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "584baebf-0808-4f20-adad-c8685a1eb082" (UID: "584baebf-0808-4f20-adad-c8685a1eb082"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:19:47.615530 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.615494 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kr97\" (UniqueName: \"kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-kube-api-access-4kr97\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:19:47.615530 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.615525 2571 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-bound-sa-token\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:19:47.615746 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.615540 2571 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/584baebf-0808-4f20-adad-c8685a1eb082-image-registry-private-configuration\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:19:47.615746 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.615554 2571 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/584baebf-0808-4f20-adad-c8685a1eb082-registry-certificates\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:19:47.615746 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.615567 2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/584baebf-0808-4f20-adad-c8685a1eb082-trusted-ca\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:19:47.615746 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.615581 2571 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/584baebf-0808-4f20-adad-c8685a1eb082-installation-pull-secrets\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:19:47.615746 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.615594 2571 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/584baebf-0808-4f20-adad-c8685a1eb082-ca-trust-extracted\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:19:47.615746 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.615605 2571 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/584baebf-0808-4f20-adad-c8685a1eb082-registry-tls\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:19:47.786964 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.786926 2571 generic.go:356] "Generic (PLEG): container finished" podID="584baebf-0808-4f20-adad-c8685a1eb082" containerID="6aee7e75c393be24f1384019409bb6df7fb8f60f994a5301e4ec3166b3fcccdd" exitCode=0 Jul 14 11:19:47.787402 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.786991 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-559449c858-hbtxv" Jul 14 11:19:47.787402 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.787019 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-559449c858-hbtxv" event={"ID":"584baebf-0808-4f20-adad-c8685a1eb082","Type":"ContainerDied","Data":"6aee7e75c393be24f1384019409bb6df7fb8f60f994a5301e4ec3166b3fcccdd"} Jul 14 11:19:47.787402 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.787082 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-559449c858-hbtxv" event={"ID":"584baebf-0808-4f20-adad-c8685a1eb082","Type":"ContainerDied","Data":"ff9b0eb055caba0abfe0b9f8f14fa6b125ca693f6fb9c1ed8bbbf0def8a1fb98"} Jul 14 11:19:47.787402 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.787114 2571 scope.go:117] "RemoveContainer" containerID="6aee7e75c393be24f1384019409bb6df7fb8f60f994a5301e4ec3166b3fcccdd" Jul 14 11:19:47.788933 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.788869 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6454d4ff44-7cxct" event={"ID":"4af9286d-a261-4209-8290-59e9ba383bfb","Type":"ContainerStarted","Data":"1a91116a15f09a7c3dc07c9f201368700a435c6d825585caab7e478eb1876378"} Jul 14 11:19:47.799956 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.798424 2571 scope.go:117] "RemoveContainer" containerID="6aee7e75c393be24f1384019409bb6df7fb8f60f994a5301e4ec3166b3fcccdd" Jul 14 11:19:47.799956 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:19:47.798901 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6aee7e75c393be24f1384019409bb6df7fb8f60f994a5301e4ec3166b3fcccdd\": container with ID starting with 6aee7e75c393be24f1384019409bb6df7fb8f60f994a5301e4ec3166b3fcccdd not found: ID does not exist" containerID="6aee7e75c393be24f1384019409bb6df7fb8f60f994a5301e4ec3166b3fcccdd" Jul 14 11:19:47.799956 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.798936 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6aee7e75c393be24f1384019409bb6df7fb8f60f994a5301e4ec3166b3fcccdd"} err="failed to get container status \"6aee7e75c393be24f1384019409bb6df7fb8f60f994a5301e4ec3166b3fcccdd\": rpc error: code = NotFound desc = could not find container \"6aee7e75c393be24f1384019409bb6df7fb8f60f994a5301e4ec3166b3fcccdd\": container with ID starting with 6aee7e75c393be24f1384019409bb6df7fb8f60f994a5301e4ec3166b3fcccdd not found: ID does not exist" Jul 14 11:19:47.814870 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.814839 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-559449c858-hbtxv"] Jul 14 11:19:47.818436 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:47.818412 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-559449c858-hbtxv"] Jul 14 11:19:48.403759 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:48.403724 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="584baebf-0808-4f20-adad-c8685a1eb082" path="/var/lib/kubelet/pods/584baebf-0808-4f20-adad-c8685a1eb082/volumes" Jul 14 11:19:50.806487 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:50.806404 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6454d4ff44-7cxct" event={"ID":"4af9286d-a261-4209-8290-59e9ba383bfb","Type":"ContainerStarted","Data":"a7cf432a4cf510638e1cd54d1f93e2073d414eb7968cd26962d39063f856a33e"} Jul 14 11:19:50.829909 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:50.829856 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6454d4ff44-7cxct" podStartSLOduration=1.598198226 podStartE2EDuration="4.829834598s" totalImagesPullingTime="3.231636372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:19:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:50.829434876 +0000 UTC m=+84.990511347" watchObservedRunningTime="2026-07-14 11:19:50.829834598 +0000 UTC m=+84.990911067" Jul 14 11:19:51.513321 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.513287 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-859c87c5bf-4n4tm"] Jul 14 11:19:51.513670 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.513653 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="584baebf-0808-4f20-adad-c8685a1eb082" containerName="registry" Jul 14 11:19:51.513756 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.513673 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="584baebf-0808-4f20-adad-c8685a1eb082" containerName="registry" Jul 14 11:19:51.513808 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.513779 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="584baebf-0808-4f20-adad-c8685a1eb082" containerName="registry" Jul 14 11:19:51.516027 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.516006 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.524724 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.524704 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 14 11:19:51.528889 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.528867 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-859c87c5bf-4n4tm"] Jul 14 11:19:51.536050 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.535979 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:51.656874 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.656838 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a2c4f67-83c6-426b-b8bb-891727248b0d-console-serving-cert\") pod \"console-859c87c5bf-4n4tm\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.657050 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.656901 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a2c4f67-83c6-426b-b8bb-891727248b0d-console-oauth-config\") pod \"console-859c87c5bf-4n4tm\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.657050 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.656950 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-service-ca\") pod \"console-859c87c5bf-4n4tm\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.657050 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.656986 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-trusted-ca-bundle\") pod \"console-859c87c5bf-4n4tm\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.657050 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.657012 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lksn\" (UniqueName: \"kubernetes.io/projected/3a2c4f67-83c6-426b-b8bb-891727248b0d-kube-api-access-9lksn\") pod \"console-859c87c5bf-4n4tm\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.657050 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.657047 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-oauth-serving-cert\") pod \"console-859c87c5bf-4n4tm\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.657341 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.657084 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-console-config\") pod \"console-859c87c5bf-4n4tm\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.758913 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.758873 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a2c4f67-83c6-426b-b8bb-891727248b0d-console-serving-cert\") pod \"console-859c87c5bf-4n4tm\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.759093 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.758940 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a2c4f67-83c6-426b-b8bb-891727248b0d-console-oauth-config\") pod \"console-859c87c5bf-4n4tm\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.759093 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.758975 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-service-ca\") pod \"console-859c87c5bf-4n4tm\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.759093 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.759010 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-trusted-ca-bundle\") pod \"console-859c87c5bf-4n4tm\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.759093 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.759038 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lksn\" (UniqueName: \"kubernetes.io/projected/3a2c4f67-83c6-426b-b8bb-891727248b0d-kube-api-access-9lksn\") pod \"console-859c87c5bf-4n4tm\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.759093 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.759079 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-oauth-serving-cert\") pod \"console-859c87c5bf-4n4tm\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.759374 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.759124 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-console-config\") pod \"console-859c87c5bf-4n4tm\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.760197 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.760165 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-service-ca\") pod \"console-859c87c5bf-4n4tm\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.760341 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.760194 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-console-config\") pod \"console-859c87c5bf-4n4tm\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.760341 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.760169 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-oauth-serving-cert\") pod \"console-859c87c5bf-4n4tm\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.760448 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.760364 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-trusted-ca-bundle\") pod \"console-859c87c5bf-4n4tm\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.761992 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.761969 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a2c4f67-83c6-426b-b8bb-891727248b0d-console-serving-cert\") pod \"console-859c87c5bf-4n4tm\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.762201 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.762181 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a2c4f67-83c6-426b-b8bb-891727248b0d-console-oauth-config\") pod \"console-859c87c5bf-4n4tm\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.768667 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.768588 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lksn\" (UniqueName: \"kubernetes.io/projected/3a2c4f67-83c6-426b-b8bb-891727248b0d-kube-api-access-9lksn\") pod \"console-859c87c5bf-4n4tm\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.826120 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.826078 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:19:51.995221 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:51.994965 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-859c87c5bf-4n4tm"] Jul 14 11:19:52.147436 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:52.147387 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:53.201495 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:53.201458 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:53.305154 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:53.305105 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:55.969978 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:55.969941 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:56.070902 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:56.070865 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:56.310095 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:56.310006 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:56.410689 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:56.410656 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:56.745260 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:56.745198 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:56.842953 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:56.842922 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:57.020933 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:57.020835 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:57.020933 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:57.020903 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:57.026422 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:57.026399 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:57.035749 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:57.035721 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:57.139567 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:57.139529 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:19:57.317159 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:57.317068 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:57.317159 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:57.317145 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:19:57.840004 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:57.839976 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:19:59.127330 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:19:59.127295 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a2c4f67_83c6_426b_b8bb_891727248b0d.slice/crio-4da66566c7840d10eeed8c760c1656ef28fb5d869f52d877d7d543c53c914bfb WatchSource:0}: Error finding container 4da66566c7840d10eeed8c760c1656ef28fb5d869f52d877d7d543c53c914bfb: Status 404 returned error can't find the container with id 4da66566c7840d10eeed8c760c1656ef28fb5d869f52d877d7d543c53c914bfb Jul 14 11:19:59.846671 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:59.846401 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-9h6fb" event={"ID":"0fe38157-e3ad-4fdb-a833-8418983f208f","Type":"ContainerStarted","Data":"8cd367bb74cb464b6bc3d1d601f85efce81e4035648d671fe350f9285d573796"} Jul 14 11:19:59.846671 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:59.846466 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-9h6fb" Jul 14 11:19:59.848568 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:59.848485 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-859c87c5bf-4n4tm" event={"ID":"3a2c4f67-83c6-426b-b8bb-891727248b0d","Type":"ContainerStarted","Data":"83eac1030e85b5178caa3ef99defedb799dc67af23f03dc079154ef28a5b13b3"} Jul 14 11:19:59.848568 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:59.848536 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-859c87c5bf-4n4tm" event={"ID":"3a2c4f67-83c6-426b-b8bb-891727248b0d","Type":"ContainerStarted","Data":"4da66566c7840d10eeed8c760c1656ef28fb5d869f52d877d7d543c53c914bfb"} Jul 14 11:19:59.867494 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:59.867466 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-9h6fb" Jul 14 11:19:59.887915 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:59.887859 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-9h6fb" podStartSLOduration=6.574301445 podStartE2EDuration="24.8878423s" totalImagesPullingTime="18.313540855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:19:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:59.86700629 +0000 UTC m=+94.028082758" watchObservedRunningTime="2026-07-14 11:19:59.8878423 +0000 UTC m=+94.048918769" Jul 14 11:19:59.888185 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:19:59.888146 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-859c87c5bf-4n4tm" podStartSLOduration=8.888131456 podStartE2EDuration="8.888131456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:19:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:19:59.887585751 +0000 UTC m=+94.048662218" watchObservedRunningTime="2026-07-14 11:19:59.888131456 +0000 UTC m=+94.049207924" Jul 14 11:20:01.826808 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:01.826766 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:20:01.827239 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:01.826844 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:20:01.832278 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:01.832252 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:20:01.860980 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:01.860952 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:20:01.909900 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:01.909861 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6454d4ff44-7cxct"] Jul 14 11:20:03.524740 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:03.524704 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:20:03.627209 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:03.627152 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:20:04.315787 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:04.315732 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:20:04.418467 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:04.418423 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:20:05.128470 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:05.128427 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:20:05.195996 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:05.195963 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:20:05.911648 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:05.911599 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:20:06.614512 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:06.614484 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-9g65d_4f499a28-6bfb-4628-9b11-3553c93a1a7b/kube-state-metrics/0.log" Jul 14 11:20:06.811936 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:06.811907 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-9g65d_4f499a28-6bfb-4628-9b11-3553c93a1a7b/kube-rbac-proxy-main/0.log" Jul 14 11:20:07.011593 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:07.011511 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-9g65d_4f499a28-6bfb-4628-9b11-3553c93a1a7b/kube-rbac-proxy-self/0.log" Jul 14 11:20:07.087294 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:07.087260 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:20:07.205737 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:07.205671 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:20:07.211705 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:07.211683 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6f589fd54-c6cfb_1edfd667-50c9-4c39-9924-6770649f9c4a/metrics-server/0.log" Jul 14 11:20:07.292506 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:07.292428 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:20:08.211772 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:08.211742 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n48lw_d6f473b7-1178-4f03-b497-7f507c0d54ff/init-textfile/0.log" Jul 14 11:20:08.412024 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:08.411999 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n48lw_d6f473b7-1178-4f03-b497-7f507c0d54ff/node-exporter/0.log" Jul 14 11:20:08.611949 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:08.611924 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n48lw_d6f473b7-1178-4f03-b497-7f507c0d54ff/kube-rbac-proxy/0.log" Jul 14 11:20:11.412752 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:11.412726 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-f55mq_7221cd75-ecfe-4220-9716-a1ada0b79d76/prometheus-operator/0.log" Jul 14 11:20:11.611359 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:11.611334 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-f55mq_7221cd75-ecfe-4220-9716-a1ada0b79d76/kube-rbac-proxy/0.log" Jul 14 11:20:12.014099 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:12.014053 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:20:12.068100 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:12.068073 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:20:12.767417 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:12.767389 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9hcgm" Jul 14 11:20:13.211595 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:13.211549 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-g6pxd_266bd207-c148-4d87-8eb7-baa4f02aa433/networking-console-plugin/0.log" Jul 14 11:20:13.811864 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:13.811826 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6454d4ff44-7cxct_4af9286d-a261-4209-8290-59e9ba383bfb/console/0.log" Jul 14 11:20:14.011668 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:14.011618 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-859c87c5bf-4n4tm_3a2c4f67-83c6-426b-b8bb-891727248b0d/console/0.log" Jul 14 11:20:14.213099 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:14.213069 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-9h6fb_0fe38157-e3ad-4fdb-a833-8418983f208f/download-server/0.log" Jul 14 11:20:15.011076 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:15.011046 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z9xv5_e142c698-e9cd-4a19-943f-74132dd73b2f/serve-healthcheck-canary/0.log" Jul 14 11:20:17.323338 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:17.323307 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:20:17.327742 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:17.327715 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6f589fd54-c6cfb" Jul 14 11:20:26.935870 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:26.935807 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6454d4ff44-7cxct" podUID="4af9286d-a261-4209-8290-59e9ba383bfb" containerName="console" containerID="cri-o://a7cf432a4cf510638e1cd54d1f93e2073d414eb7968cd26962d39063f856a33e" gracePeriod=15 Jul 14 11:20:27.196447 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.196389 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:20:27.268309 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.268268 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnmlt\" (UniqueName: \"kubernetes.io/projected/4af9286d-a261-4209-8290-59e9ba383bfb-kube-api-access-qnmlt\") pod \"4af9286d-a261-4209-8290-59e9ba383bfb\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " Jul 14 11:20:27.268309 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.268311 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4af9286d-a261-4209-8290-59e9ba383bfb-service-ca\") pod \"4af9286d-a261-4209-8290-59e9ba383bfb\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " Jul 14 11:20:27.268519 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.268344 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4af9286d-a261-4209-8290-59e9ba383bfb-console-config\") pod \"4af9286d-a261-4209-8290-59e9ba383bfb\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " Jul 14 11:20:27.268519 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.268363 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4af9286d-a261-4209-8290-59e9ba383bfb-console-serving-cert\") pod \"4af9286d-a261-4209-8290-59e9ba383bfb\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " Jul 14 11:20:27.268519 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.268504 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4af9286d-a261-4209-8290-59e9ba383bfb-console-oauth-config\") pod \"4af9286d-a261-4209-8290-59e9ba383bfb\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " Jul 14 11:20:27.268654 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.268577 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4af9286d-a261-4209-8290-59e9ba383bfb-oauth-serving-cert\") pod \"4af9286d-a261-4209-8290-59e9ba383bfb\" (UID: \"4af9286d-a261-4209-8290-59e9ba383bfb\") " Jul 14 11:20:27.268812 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.268777 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4af9286d-a261-4209-8290-59e9ba383bfb-service-ca" (OuterVolumeSpecName: "service-ca") pod "4af9286d-a261-4209-8290-59e9ba383bfb" (UID: "4af9286d-a261-4209-8290-59e9ba383bfb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:20:27.268812 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.268792 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4af9286d-a261-4209-8290-59e9ba383bfb-console-config" (OuterVolumeSpecName: "console-config") pod "4af9286d-a261-4209-8290-59e9ba383bfb" (UID: "4af9286d-a261-4209-8290-59e9ba383bfb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:20:27.268929 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.268880 2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4af9286d-a261-4209-8290-59e9ba383bfb-service-ca\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:20:27.269017 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.268900 2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4af9286d-a261-4209-8290-59e9ba383bfb-console-config\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:20:27.269082 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.269063 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4af9286d-a261-4209-8290-59e9ba383bfb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4af9286d-a261-4209-8290-59e9ba383bfb" (UID: "4af9286d-a261-4209-8290-59e9ba383bfb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:20:27.270740 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.270719 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4af9286d-a261-4209-8290-59e9ba383bfb-kube-api-access-qnmlt" (OuterVolumeSpecName: "kube-api-access-qnmlt") pod "4af9286d-a261-4209-8290-59e9ba383bfb" (UID: "4af9286d-a261-4209-8290-59e9ba383bfb"). InnerVolumeSpecName "kube-api-access-qnmlt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:20:27.270835 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.270766 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4af9286d-a261-4209-8290-59e9ba383bfb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4af9286d-a261-4209-8290-59e9ba383bfb" (UID: "4af9286d-a261-4209-8290-59e9ba383bfb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:20:27.270835 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.270817 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4af9286d-a261-4209-8290-59e9ba383bfb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4af9286d-a261-4209-8290-59e9ba383bfb" (UID: "4af9286d-a261-4209-8290-59e9ba383bfb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:20:27.370306 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.370274 2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4af9286d-a261-4209-8290-59e9ba383bfb-oauth-serving-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:20:27.370306 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.370297 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnmlt\" (UniqueName: \"kubernetes.io/projected/4af9286d-a261-4209-8290-59e9ba383bfb-kube-api-access-qnmlt\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:20:27.370306 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.370308 2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4af9286d-a261-4209-8290-59e9ba383bfb-console-serving-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:20:27.370306 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.370317 2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4af9286d-a261-4209-8290-59e9ba383bfb-console-oauth-config\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:20:27.947833 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.947800 2571 generic.go:356] "Generic (PLEG): container finished" podID="4af9286d-a261-4209-8290-59e9ba383bfb" containerID="a7cf432a4cf510638e1cd54d1f93e2073d414eb7968cd26962d39063f856a33e" exitCode=0 Jul 14 11:20:27.948223 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.947842 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6454d4ff44-7cxct" event={"ID":"4af9286d-a261-4209-8290-59e9ba383bfb","Type":"ContainerDied","Data":"a7cf432a4cf510638e1cd54d1f93e2073d414eb7968cd26962d39063f856a33e"} Jul 14 11:20:27.948223 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.947859 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6454d4ff44-7cxct" Jul 14 11:20:27.948223 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.947883 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6454d4ff44-7cxct" event={"ID":"4af9286d-a261-4209-8290-59e9ba383bfb","Type":"ContainerDied","Data":"1a91116a15f09a7c3dc07c9f201368700a435c6d825585caab7e478eb1876378"} Jul 14 11:20:27.948223 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.947903 2571 scope.go:117] "RemoveContainer" containerID="a7cf432a4cf510638e1cd54d1f93e2073d414eb7968cd26962d39063f856a33e" Jul 14 11:20:27.957281 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.956596 2571 scope.go:117] "RemoveContainer" containerID="a7cf432a4cf510638e1cd54d1f93e2073d414eb7968cd26962d39063f856a33e" Jul 14 11:20:27.957506 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:20:27.957485 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7cf432a4cf510638e1cd54d1f93e2073d414eb7968cd26962d39063f856a33e\": container with ID starting with a7cf432a4cf510638e1cd54d1f93e2073d414eb7968cd26962d39063f856a33e not found: ID does not exist" containerID="a7cf432a4cf510638e1cd54d1f93e2073d414eb7968cd26962d39063f856a33e" Jul 14 11:20:27.957577 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.957516 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7cf432a4cf510638e1cd54d1f93e2073d414eb7968cd26962d39063f856a33e"} err="failed to get container status \"a7cf432a4cf510638e1cd54d1f93e2073d414eb7968cd26962d39063f856a33e\": rpc error: code = NotFound desc = could not find container \"a7cf432a4cf510638e1cd54d1f93e2073d414eb7968cd26962d39063f856a33e\": container with ID starting with a7cf432a4cf510638e1cd54d1f93e2073d414eb7968cd26962d39063f856a33e not found: ID does not exist" Jul 14 11:20:27.970309 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.970283 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6454d4ff44-7cxct"] Jul 14 11:20:27.974237 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:27.974215 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6454d4ff44-7cxct"] Jul 14 11:20:28.402244 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:28.402215 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4af9286d-a261-4209-8290-59e9ba383bfb" path="/var/lib/kubelet/pods/4af9286d-a261-4209-8290-59e9ba383bfb/volumes" Jul 14 11:20:33.055984 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:33.055939 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:20:51.861677 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:51.861622 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:20:52.057451 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:52.057419 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:20:54.528527 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.528496 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 11:20:54.529052 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.528874 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4af9286d-a261-4209-8290-59e9ba383bfb" containerName="console" Jul 14 11:20:54.529052 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.528894 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4af9286d-a261-4209-8290-59e9ba383bfb" containerName="console" Jul 14 11:20:54.529052 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.528970 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4af9286d-a261-4209-8290-59e9ba383bfb" containerName="console" Jul 14 11:20:54.531609 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.531585 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.534555 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.534536 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 14 11:20:54.534845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.534819 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 14 11:20:54.534845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.534842 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 14 11:20:54.535022 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.534848 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 14 11:20:54.535022 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.534872 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 14 11:20:54.535022 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.534829 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 14 11:20:54.535022 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.534825 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-s4x5m\"" Jul 14 11:20:54.535022 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.534874 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 14 11:20:54.535310 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.535294 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 14 11:20:54.540669 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.540464 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 14 11:20:54.544996 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.544970 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 11:20:54.575898 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.575866 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d8fb55f-acfd-4179-a638-bf87cfb05545-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.575898 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.575902 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4d8fb55f-acfd-4179-a638-bf87cfb05545-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.576080 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.575920 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4d8fb55f-acfd-4179-a638-bf87cfb05545-config-out\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.576080 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.575941 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d8fb55f-acfd-4179-a638-bf87cfb05545-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.576080 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.575995 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d8fb55f-acfd-4179-a638-bf87cfb05545-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.576080 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.576046 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4d8fb55f-acfd-4179-a638-bf87cfb05545-config-volume\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.576080 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.576069 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d8fb55f-acfd-4179-a638-bf87cfb05545-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.576237 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.576085 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljf74\" (UniqueName: \"kubernetes.io/projected/4d8fb55f-acfd-4179-a638-bf87cfb05545-kube-api-access-ljf74\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.576237 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.576117 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4d8fb55f-acfd-4179-a638-bf87cfb05545-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.576237 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.576159 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4d8fb55f-acfd-4179-a638-bf87cfb05545-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.576237 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.576186 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4d8fb55f-acfd-4179-a638-bf87cfb05545-web-config\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.576237 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.576209 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4d8fb55f-acfd-4179-a638-bf87cfb05545-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.576237 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.576231 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4d8fb55f-acfd-4179-a638-bf87cfb05545-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.677469 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.677427 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4d8fb55f-acfd-4179-a638-bf87cfb05545-config-volume\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.677469 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.677461 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d8fb55f-acfd-4179-a638-bf87cfb05545-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.677469 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.677478 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljf74\" (UniqueName: \"kubernetes.io/projected/4d8fb55f-acfd-4179-a638-bf87cfb05545-kube-api-access-ljf74\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.677823 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.677499 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4d8fb55f-acfd-4179-a638-bf87cfb05545-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.677823 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.677520 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4d8fb55f-acfd-4179-a638-bf87cfb05545-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.677823 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.677552 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4d8fb55f-acfd-4179-a638-bf87cfb05545-web-config\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.677823 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.677578 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4d8fb55f-acfd-4179-a638-bf87cfb05545-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.677823 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.677606 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4d8fb55f-acfd-4179-a638-bf87cfb05545-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.677823 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.677653 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d8fb55f-acfd-4179-a638-bf87cfb05545-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.677823 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.677678 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4d8fb55f-acfd-4179-a638-bf87cfb05545-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.678174 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.678115 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4d8fb55f-acfd-4179-a638-bf87cfb05545-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.678535 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.678510 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d8fb55f-acfd-4179-a638-bf87cfb05545-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.678655 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.678567 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4d8fb55f-acfd-4179-a638-bf87cfb05545-config-out\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.678655 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.678599 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d8fb55f-acfd-4179-a638-bf87cfb05545-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.678778 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.678661 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d8fb55f-acfd-4179-a638-bf87cfb05545-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.678778 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.678743 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d8fb55f-acfd-4179-a638-bf87cfb05545-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.680561 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.680528 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4d8fb55f-acfd-4179-a638-bf87cfb05545-web-config\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.680710 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.680683 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4d8fb55f-acfd-4179-a638-bf87cfb05545-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.680789 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.680765 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4d8fb55f-acfd-4179-a638-bf87cfb05545-config-volume\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.680983 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.680940 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4d8fb55f-acfd-4179-a638-bf87cfb05545-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.681297 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.681276 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d8fb55f-acfd-4179-a638-bf87cfb05545-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.681400 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.681381 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4d8fb55f-acfd-4179-a638-bf87cfb05545-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.681525 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.681501 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4d8fb55f-acfd-4179-a638-bf87cfb05545-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.681617 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.681600 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4d8fb55f-acfd-4179-a638-bf87cfb05545-config-out\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.682723 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.682704 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d8fb55f-acfd-4179-a638-bf87cfb05545-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.686439 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.686419 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljf74\" (UniqueName: \"kubernetes.io/projected/4d8fb55f-acfd-4179-a638-bf87cfb05545-kube-api-access-ljf74\") pod \"alertmanager-main-0\" (UID: \"4d8fb55f-acfd-4179-a638-bf87cfb05545\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.842849 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.842741 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 11:20:54.978399 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:54.978376 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 11:20:54.980230 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:20:54.980198 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d8fb55f_acfd_4179_a638_bf87cfb05545.slice/crio-3cb79e59002fc90769616303b2c9521deea413c1151e50a1dcfadaf6a2f238f2 WatchSource:0}: Error finding container 3cb79e59002fc90769616303b2c9521deea413c1151e50a1dcfadaf6a2f238f2: Status 404 returned error can't find the container with id 3cb79e59002fc90769616303b2c9521deea413c1151e50a1dcfadaf6a2f238f2 Jul 14 11:20:55.041536 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:55.041491 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d8fb55f-acfd-4179-a638-bf87cfb05545","Type":"ContainerStarted","Data":"3cb79e59002fc90769616303b2c9521deea413c1151e50a1dcfadaf6a2f238f2"} Jul 14 11:20:56.938617 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:56.938581 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s"] Jul 14 11:20:56.941969 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:56.941931 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:56.944837 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:56.944808 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 14 11:20:56.944987 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:56.944812 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 14 11:20:56.944987 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:56.944844 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 14 11:20:56.944987 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:56.944880 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 14 11:20:56.945347 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:56.945331 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-gpckk\"" Jul 14 11:20:56.945432 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:56.945369 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 14 11:20:56.953878 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:56.953849 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 14 11:20:56.956688 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:56.956661 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s"] Jul 14 11:20:56.993972 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:56.993925 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3475c11c-6707-4978-9b57-538d487b9806-secret-telemeter-client\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:56.994144 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:56.993978 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gztv9\" (UniqueName: \"kubernetes.io/projected/3475c11c-6707-4978-9b57-538d487b9806-kube-api-access-gztv9\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:56.994144 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:56.994014 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3475c11c-6707-4978-9b57-538d487b9806-telemeter-client-tls\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:56.994144 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:56.994039 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3475c11c-6707-4978-9b57-538d487b9806-metrics-client-ca\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:56.994144 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:56.994065 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3475c11c-6707-4978-9b57-538d487b9806-serving-certs-ca-bundle\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:56.994324 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:56.994149 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3475c11c-6707-4978-9b57-538d487b9806-federate-client-tls\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:56.994324 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:56.994181 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3475c11c-6707-4978-9b57-538d487b9806-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:56.994324 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:56.994213 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3475c11c-6707-4978-9b57-538d487b9806-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:57.052013 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:57.051972 2571 generic.go:356] "Generic (PLEG): container finished" podID="4d8fb55f-acfd-4179-a638-bf87cfb05545" containerID="f806f0c7f805b401c81556d7fc594da4779395b6cea7b132b325b133c8e03cef" exitCode=0 Jul 14 11:20:57.052175 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:57.052067 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d8fb55f-acfd-4179-a638-bf87cfb05545","Type":"ContainerDied","Data":"f806f0c7f805b401c81556d7fc594da4779395b6cea7b132b325b133c8e03cef"} Jul 14 11:20:57.094880 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:57.094838 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3475c11c-6707-4978-9b57-538d487b9806-federate-client-tls\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:57.094880 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:57.094879 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3475c11c-6707-4978-9b57-538d487b9806-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:57.095152 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:57.094977 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3475c11c-6707-4978-9b57-538d487b9806-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:57.095152 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:57.095037 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3475c11c-6707-4978-9b57-538d487b9806-secret-telemeter-client\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:57.095152 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:57.095063 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gztv9\" (UniqueName: \"kubernetes.io/projected/3475c11c-6707-4978-9b57-538d487b9806-kube-api-access-gztv9\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:57.095152 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:57.095116 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3475c11c-6707-4978-9b57-538d487b9806-telemeter-client-tls\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:57.095152 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:57.095141 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3475c11c-6707-4978-9b57-538d487b9806-metrics-client-ca\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:57.095472 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:57.095161 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3475c11c-6707-4978-9b57-538d487b9806-serving-certs-ca-bundle\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:57.096349 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:57.096068 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3475c11c-6707-4978-9b57-538d487b9806-serving-certs-ca-bundle\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:57.096349 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:57.096101 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3475c11c-6707-4978-9b57-538d487b9806-metrics-client-ca\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:57.099102 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:57.096528 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3475c11c-6707-4978-9b57-538d487b9806-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:57.099102 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:57.097909 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3475c11c-6707-4978-9b57-538d487b9806-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:57.099102 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:57.098046 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3475c11c-6707-4978-9b57-538d487b9806-federate-client-tls\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:57.099279 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:57.099213 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3475c11c-6707-4978-9b57-538d487b9806-secret-telemeter-client\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:57.099590 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:57.099571 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3475c11c-6707-4978-9b57-538d487b9806-telemeter-client-tls\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:57.103339 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:57.103316 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gztv9\" (UniqueName: \"kubernetes.io/projected/3475c11c-6707-4978-9b57-538d487b9806-kube-api-access-gztv9\") pod \"telemeter-client-85cb9b85bd-f7g4s\" (UID: \"3475c11c-6707-4978-9b57-538d487b9806\") " pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:57.258484 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:57.258166 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" Jul 14 11:20:57.434519 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:57.434484 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s"] Jul 14 11:20:57.436607 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:20:57.436581 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3475c11c_6707_4978_9b57_538d487b9806.slice/crio-3a4cd23a158478bb9315c7e464f4f7d9c84c6a13fb82533b137d5bfaee29ed01 WatchSource:0}: Error finding container 3a4cd23a158478bb9315c7e464f4f7d9c84c6a13fb82533b137d5bfaee29ed01: Status 404 returned error can't find the container with id 3a4cd23a158478bb9315c7e464f4f7d9c84c6a13fb82533b137d5bfaee29ed01 Jul 14 11:20:58.058288 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:58.058253 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" event={"ID":"3475c11c-6707-4978-9b57-538d487b9806","Type":"ContainerStarted","Data":"3a4cd23a158478bb9315c7e464f4f7d9c84c6a13fb82533b137d5bfaee29ed01"} Jul 14 11:20:59.065903 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:59.065868 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d8fb55f-acfd-4179-a638-bf87cfb05545","Type":"ContainerStarted","Data":"c4b0a5b4515a8233c25c7681bc0e88e3575d8abc0145f978685a173320ab1a3d"} Jul 14 11:20:59.066296 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:59.065955 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d8fb55f-acfd-4179-a638-bf87cfb05545","Type":"ContainerStarted","Data":"36822cb617ae8908d37ab1cf23d0fe1fa270f20599c2f3fc32bf20c6deb616ad"} Jul 14 11:20:59.066296 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:59.065987 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d8fb55f-acfd-4179-a638-bf87cfb05545","Type":"ContainerStarted","Data":"ff665a5ee90ea8a1f26e3f2a59cff23dc1e0a0ebdf0c80b7ba78a0ff10fa4e37"} Jul 14 11:20:59.066296 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:20:59.066011 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d8fb55f-acfd-4179-a638-bf87cfb05545","Type":"ContainerStarted","Data":"2709f0e8a2ecc2bd1f93377f53fb2893f38c49686d36a3cd8e85471cdad9db62"} Jul 14 11:21:00.074533 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:00.074497 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d8fb55f-acfd-4179-a638-bf87cfb05545","Type":"ContainerStarted","Data":"fa5bbd28a775f37e93260e931350a871c87e5aaa76efd248945b5b083477ce53"} Jul 14 11:21:00.076658 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:00.076611 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" event={"ID":"3475c11c-6707-4978-9b57-538d487b9806","Type":"ContainerStarted","Data":"542db227e08cffba19b40f2f54f8c2179a0cc6f7ad5cf845c8155cac6a10ddd5"} Jul 14 11:21:00.076803 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:00.076693 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" event={"ID":"3475c11c-6707-4978-9b57-538d487b9806","Type":"ContainerStarted","Data":"e9046f3dbf867fd71ee73bc211683c98c71d4a259c443b992ccaa9f16212b0cc"} Jul 14 11:21:00.076803 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:00.076727 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" event={"ID":"3475c11c-6707-4978-9b57-538d487b9806","Type":"ContainerStarted","Data":"25e086afaeaa84339981eb6d2bd7653ba59d7ce12f427a5740bb5324388336bf"} Jul 14 11:21:00.101392 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:00.101152 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-85cb9b85bd-f7g4s" podStartSLOduration=2.23944184 podStartE2EDuration="4.101129269s" totalImagesPullingTime="1.861687429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:20:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:21:00.100472504 +0000 UTC m=+154.261548972" watchObservedRunningTime="2026-07-14 11:21:00.101129269 +0000 UTC m=+154.262205736" Jul 14 11:21:00.886203 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:00.886168 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-566ddbbf98-z4kql"] Jul 14 11:21:00.888445 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:00.888427 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:00.902332 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:00.902308 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-566ddbbf98-z4kql"] Jul 14 11:21:00.927159 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:00.927127 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70b71177-48b9-4fd8-a350-268a3e702f14-console-serving-cert\") pod \"console-566ddbbf98-z4kql\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:00.927314 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:00.927170 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-console-config\") pod \"console-566ddbbf98-z4kql\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:00.927314 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:00.927197 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-trusted-ca-bundle\") pod \"console-566ddbbf98-z4kql\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:00.927314 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:00.927292 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-oauth-serving-cert\") pod \"console-566ddbbf98-z4kql\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:00.927433 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:00.927349 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-service-ca\") pod \"console-566ddbbf98-z4kql\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:00.927433 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:00.927393 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70b71177-48b9-4fd8-a350-268a3e702f14-console-oauth-config\") pod \"console-566ddbbf98-z4kql\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:00.927433 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:00.927419 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q74vh\" (UniqueName: \"kubernetes.io/projected/70b71177-48b9-4fd8-a350-268a3e702f14-kube-api-access-q74vh\") pod \"console-566ddbbf98-z4kql\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:00.937920 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:00.937890 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:21:01.027821 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:01.027786 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-oauth-serving-cert\") pod \"console-566ddbbf98-z4kql\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:01.027821 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:01.027829 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-service-ca\") pod \"console-566ddbbf98-z4kql\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:01.028033 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:01.027855 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70b71177-48b9-4fd8-a350-268a3e702f14-console-oauth-config\") pod \"console-566ddbbf98-z4kql\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:01.028033 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:01.027871 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q74vh\" (UniqueName: \"kubernetes.io/projected/70b71177-48b9-4fd8-a350-268a3e702f14-kube-api-access-q74vh\") pod \"console-566ddbbf98-z4kql\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:01.028033 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:01.027959 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70b71177-48b9-4fd8-a350-268a3e702f14-console-serving-cert\") pod \"console-566ddbbf98-z4kql\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:01.028033 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:01.028022 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-console-config\") pod \"console-566ddbbf98-z4kql\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:01.028240 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:01.028067 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-trusted-ca-bundle\") pod \"console-566ddbbf98-z4kql\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:01.028746 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:01.028718 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-oauth-serving-cert\") pod \"console-566ddbbf98-z4kql\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:01.028888 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:01.028771 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-console-config\") pod \"console-566ddbbf98-z4kql\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:01.028941 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:01.028889 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-trusted-ca-bundle\") pod \"console-566ddbbf98-z4kql\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:01.029108 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:01.029092 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-service-ca\") pod \"console-566ddbbf98-z4kql\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:01.031412 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:01.030696 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70b71177-48b9-4fd8-a350-268a3e702f14-console-oauth-config\") pod \"console-566ddbbf98-z4kql\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:01.031412 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:01.031387 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70b71177-48b9-4fd8-a350-268a3e702f14-console-serving-cert\") pod \"console-566ddbbf98-z4kql\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:01.036133 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:01.036109 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q74vh\" (UniqueName: \"kubernetes.io/projected/70b71177-48b9-4fd8-a350-268a3e702f14-kube-api-access-q74vh\") pod \"console-566ddbbf98-z4kql\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:01.039583 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:01.039554 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:21:01.084863 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:01.084821 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4d8fb55f-acfd-4179-a638-bf87cfb05545","Type":"ContainerStarted","Data":"6cf85b98082ce4e494bb094f6db6c98f3e3dadf4fd9310b8ecd8447a48dc7490"} Jul 14 11:21:01.112052 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:01.111996 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.341507272 podStartE2EDuration="7.111978089s" totalImagesPullingTime="3.753977064s" totalInitContainerRuntime="16.493753ms" isStatefulPod=false podCreationTimestamp="2026-07-14 11:20:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:21:01.110213268 +0000 UTC m=+155.271289879" watchObservedRunningTime="2026-07-14 11:21:01.111978089 +0000 UTC m=+155.273054556" Jul 14 11:21:01.197873 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:01.197778 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:01.355123 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:01.355094 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-566ddbbf98-z4kql"] Jul 14 11:21:02.089716 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:02.089674 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-566ddbbf98-z4kql" event={"ID":"70b71177-48b9-4fd8-a350-268a3e702f14","Type":"ContainerStarted","Data":"8391cc954ba5f0a562cae3c6f7a7f257e492ab5f08b1516b29576705e873b686"} Jul 14 11:21:02.090115 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:02.089754 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-566ddbbf98-z4kql" event={"ID":"70b71177-48b9-4fd8-a350-268a3e702f14","Type":"ContainerStarted","Data":"f8c251ecf19bc93b03cb68aa9fad40a0d7eb06e9c351bc0f5513fe16341fde9c"} Jul 14 11:21:02.108515 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:02.108455 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-566ddbbf98-z4kql" podStartSLOduration=2.10843655 podStartE2EDuration="2.10843655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:21:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:21:02.107824821 +0000 UTC m=+156.268901288" watchObservedRunningTime="2026-07-14 11:21:02.10843655 +0000 UTC m=+156.269513018" Jul 14 11:21:11.198715 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:11.198667 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:11.198715 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:11.198732 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:11.203364 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:11.203341 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:12.130142 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:12.130112 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:21:12.175414 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:12.175377 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-859c87c5bf-4n4tm"] Jul 14 11:21:12.282287 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:12.282256 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:21:12.384278 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:12.384173 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:21:12.501640 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:12.501593 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:21:34.126279 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:34.126237 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:21:34.181309 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:34.181271 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:21:37.198759 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.198716 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-859c87c5bf-4n4tm" podUID="3a2c4f67-83c6-426b-b8bb-891727248b0d" containerName="console" containerID="cri-o://83eac1030e85b5178caa3ef99defedb799dc67af23f03dc079154ef28a5b13b3" gracePeriod=15 Jul 14 11:21:37.434774 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.434748 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:21:37.532527 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.532442 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-service-ca\") pod \"3a2c4f67-83c6-426b-b8bb-891727248b0d\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " Jul 14 11:21:37.532527 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.532509 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-oauth-serving-cert\") pod \"3a2c4f67-83c6-426b-b8bb-891727248b0d\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " Jul 14 11:21:37.532760 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.532545 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-trusted-ca-bundle\") pod \"3a2c4f67-83c6-426b-b8bb-891727248b0d\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " Jul 14 11:21:37.532760 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.532576 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lksn\" (UniqueName: \"kubernetes.io/projected/3a2c4f67-83c6-426b-b8bb-891727248b0d-kube-api-access-9lksn\") pod \"3a2c4f67-83c6-426b-b8bb-891727248b0d\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " Jul 14 11:21:37.532760 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.532621 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a2c4f67-83c6-426b-b8bb-891727248b0d-console-oauth-config\") pod \"3a2c4f67-83c6-426b-b8bb-891727248b0d\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " Jul 14 11:21:37.532760 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.532674 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a2c4f67-83c6-426b-b8bb-891727248b0d-console-serving-cert\") pod \"3a2c4f67-83c6-426b-b8bb-891727248b0d\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " Jul 14 11:21:37.532760 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.532720 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-console-config\") pod \"3a2c4f67-83c6-426b-b8bb-891727248b0d\" (UID: \"3a2c4f67-83c6-426b-b8bb-891727248b0d\") " Jul 14 11:21:37.533007 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.532914 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-service-ca" (OuterVolumeSpecName: "service-ca") pod "3a2c4f67-83c6-426b-b8bb-891727248b0d" (UID: "3a2c4f67-83c6-426b-b8bb-891727248b0d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:21:37.533007 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.532934 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3a2c4f67-83c6-426b-b8bb-891727248b0d" (UID: "3a2c4f67-83c6-426b-b8bb-891727248b0d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:21:37.533007 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.532956 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3a2c4f67-83c6-426b-b8bb-891727248b0d" (UID: "3a2c4f67-83c6-426b-b8bb-891727248b0d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:21:37.533160 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.533017 2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-oauth-serving-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:21:37.533160 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.533034 2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-trusted-ca-bundle\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:21:37.533160 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.533046 2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-service-ca\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:21:37.533314 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.533218 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-console-config" (OuterVolumeSpecName: "console-config") pod "3a2c4f67-83c6-426b-b8bb-891727248b0d" (UID: "3a2c4f67-83c6-426b-b8bb-891727248b0d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:21:37.534930 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.534884 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a2c4f67-83c6-426b-b8bb-891727248b0d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3a2c4f67-83c6-426b-b8bb-891727248b0d" (UID: "3a2c4f67-83c6-426b-b8bb-891727248b0d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:21:37.535034 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.534909 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a2c4f67-83c6-426b-b8bb-891727248b0d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3a2c4f67-83c6-426b-b8bb-891727248b0d" (UID: "3a2c4f67-83c6-426b-b8bb-891727248b0d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:21:37.535034 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.534960 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a2c4f67-83c6-426b-b8bb-891727248b0d-kube-api-access-9lksn" (OuterVolumeSpecName: "kube-api-access-9lksn") pod "3a2c4f67-83c6-426b-b8bb-891727248b0d" (UID: "3a2c4f67-83c6-426b-b8bb-891727248b0d"). InnerVolumeSpecName "kube-api-access-9lksn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:21:37.634554 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.634511 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lksn\" (UniqueName: \"kubernetes.io/projected/3a2c4f67-83c6-426b-b8bb-891727248b0d-kube-api-access-9lksn\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:21:37.634554 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.634547 2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a2c4f67-83c6-426b-b8bb-891727248b0d-console-oauth-config\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:21:37.634554 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.634559 2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a2c4f67-83c6-426b-b8bb-891727248b0d-console-serving-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:21:37.634791 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:37.634572 2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a2c4f67-83c6-426b-b8bb-891727248b0d-console-config\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:21:38.224287 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:38.224253 2571 generic.go:356] "Generic (PLEG): container finished" podID="3a2c4f67-83c6-426b-b8bb-891727248b0d" containerID="83eac1030e85b5178caa3ef99defedb799dc67af23f03dc079154ef28a5b13b3" exitCode=0 Jul 14 11:21:38.224769 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:38.224310 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-859c87c5bf-4n4tm" Jul 14 11:21:38.224769 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:38.224338 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-859c87c5bf-4n4tm" event={"ID":"3a2c4f67-83c6-426b-b8bb-891727248b0d","Type":"ContainerDied","Data":"83eac1030e85b5178caa3ef99defedb799dc67af23f03dc079154ef28a5b13b3"} Jul 14 11:21:38.224769 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:38.224436 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-859c87c5bf-4n4tm" event={"ID":"3a2c4f67-83c6-426b-b8bb-891727248b0d","Type":"ContainerDied","Data":"4da66566c7840d10eeed8c760c1656ef28fb5d869f52d877d7d543c53c914bfb"} Jul 14 11:21:38.224769 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:38.224468 2571 scope.go:117] "RemoveContainer" containerID="83eac1030e85b5178caa3ef99defedb799dc67af23f03dc079154ef28a5b13b3" Jul 14 11:21:38.232368 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:38.232350 2571 scope.go:117] "RemoveContainer" containerID="83eac1030e85b5178caa3ef99defedb799dc67af23f03dc079154ef28a5b13b3" Jul 14 11:21:38.232608 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:21:38.232590 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83eac1030e85b5178caa3ef99defedb799dc67af23f03dc079154ef28a5b13b3\": container with ID starting with 83eac1030e85b5178caa3ef99defedb799dc67af23f03dc079154ef28a5b13b3 not found: ID does not exist" containerID="83eac1030e85b5178caa3ef99defedb799dc67af23f03dc079154ef28a5b13b3" Jul 14 11:21:38.232683 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:38.232616 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83eac1030e85b5178caa3ef99defedb799dc67af23f03dc079154ef28a5b13b3"} err="failed to get container status \"83eac1030e85b5178caa3ef99defedb799dc67af23f03dc079154ef28a5b13b3\": rpc error: code = NotFound desc = could not find container \"83eac1030e85b5178caa3ef99defedb799dc67af23f03dc079154ef28a5b13b3\": container with ID starting with 83eac1030e85b5178caa3ef99defedb799dc67af23f03dc079154ef28a5b13b3 not found: ID does not exist" Jul 14 11:21:38.245166 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:38.245123 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-859c87c5bf-4n4tm"] Jul 14 11:21:38.246321 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:38.246304 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-859c87c5bf-4n4tm"] Jul 14 11:21:38.401793 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:21:38.401763 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a2c4f67-83c6-426b-b8bb-891727248b0d" path="/var/lib/kubelet/pods/3a2c4f67-83c6-426b-b8bb-891727248b0d/volumes" Jul 14 11:22:07.019573 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:07.019535 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:22:07.081433 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:07.081405 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:22:18.919297 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:18.919237 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:22:19.019602 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:19.019545 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:22:27.018067 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:27.018032 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:22:27.124035 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:27.123992 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:22:28.746177 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:28.746146 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:22:36.005081 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.005037 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d655fdd4b-h7zqq"] Jul 14 11:22:36.005572 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.005310 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a2c4f67-83c6-426b-b8bb-891727248b0d" containerName="console" Jul 14 11:22:36.005572 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.005321 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a2c4f67-83c6-426b-b8bb-891727248b0d" containerName="console" Jul 14 11:22:36.005572 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.005383 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a2c4f67-83c6-426b-b8bb-891727248b0d" containerName="console" Jul 14 11:22:36.009349 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.009329 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.020656 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.020615 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d655fdd4b-h7zqq"] Jul 14 11:22:36.063275 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.063234 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:22:36.126525 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.126493 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:22:36.154832 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.154809 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-service-ca\") pod \"console-7d655fdd4b-h7zqq\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.154915 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.154850 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-console-serving-cert\") pod \"console-7d655fdd4b-h7zqq\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.154915 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.154903 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-oauth-serving-cert\") pod \"console-7d655fdd4b-h7zqq\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.154982 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.154938 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-console-oauth-config\") pod \"console-7d655fdd4b-h7zqq\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.155019 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.154983 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-console-config\") pod \"console-7d655fdd4b-h7zqq\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.155019 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.155000 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8bnj\" (UniqueName: \"kubernetes.io/projected/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-kube-api-access-p8bnj\") pod \"console-7d655fdd4b-h7zqq\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.155080 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.155025 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-trusted-ca-bundle\") pod \"console-7d655fdd4b-h7zqq\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.256239 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.256158 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-trusted-ca-bundle\") pod \"console-7d655fdd4b-h7zqq\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.256239 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.256199 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-service-ca\") pod \"console-7d655fdd4b-h7zqq\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.256449 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.256321 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-console-serving-cert\") pod \"console-7d655fdd4b-h7zqq\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.256449 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.256365 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-oauth-serving-cert\") pod \"console-7d655fdd4b-h7zqq\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.256449 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.256398 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-console-oauth-config\") pod \"console-7d655fdd4b-h7zqq\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.256449 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.256445 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-console-config\") pod \"console-7d655fdd4b-h7zqq\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.256669 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.256474 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8bnj\" (UniqueName: \"kubernetes.io/projected/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-kube-api-access-p8bnj\") pod \"console-7d655fdd4b-h7zqq\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.257394 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.257326 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-service-ca\") pod \"console-7d655fdd4b-h7zqq\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.257517 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.257391 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-oauth-serving-cert\") pod \"console-7d655fdd4b-h7zqq\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.257580 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.257504 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-console-config\") pod \"console-7d655fdd4b-h7zqq\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.257935 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.257908 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-trusted-ca-bundle\") pod \"console-7d655fdd4b-h7zqq\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.259768 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.259742 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-console-serving-cert\") pod \"console-7d655fdd4b-h7zqq\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.259872 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.259838 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-console-oauth-config\") pod \"console-7d655fdd4b-h7zqq\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.265402 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.265383 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8bnj\" (UniqueName: \"kubernetes.io/projected/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-kube-api-access-p8bnj\") pod \"console-7d655fdd4b-h7zqq\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.317993 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.317966 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:36.450553 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:36.450531 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d655fdd4b-h7zqq"] Jul 14 11:22:36.452503 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:22:36.452479 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c43df2b_54b8_4329_8fb7_4ab2ae4ebf77.slice/crio-cb7020a8f3090110b10e5cdab73079146d42dbda11568df510edfbe09146821b WatchSource:0}: Error finding container cb7020a8f3090110b10e5cdab73079146d42dbda11568df510edfbe09146821b: Status 404 returned error can't find the container with id cb7020a8f3090110b10e5cdab73079146d42dbda11568df510edfbe09146821b Jul 14 11:22:37.444986 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:37.444948 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d655fdd4b-h7zqq" event={"ID":"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77","Type":"ContainerStarted","Data":"dfadc71a82b8671e47612f354115648e6760c301402f5f0a618966eaa7adb15f"} Jul 14 11:22:37.445370 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:37.444996 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d655fdd4b-h7zqq" event={"ID":"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77","Type":"ContainerStarted","Data":"cb7020a8f3090110b10e5cdab73079146d42dbda11568df510edfbe09146821b"} Jul 14 11:22:37.468211 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:37.468159 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d655fdd4b-h7zqq" podStartSLOduration=2.468141659 podStartE2EDuration="2.468141659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:22:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:22:37.466325578 +0000 UTC m=+251.627402163" watchObservedRunningTime="2026-07-14 11:22:37.468141659 +0000 UTC m=+251.629218127" Jul 14 11:22:46.318464 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:46.318426 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:46.318925 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:46.318513 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:46.323040 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:46.323016 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:46.485618 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:46.485589 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:22:46.536176 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:46.536144 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-566ddbbf98-z4kql"] Jul 14 11:22:46.692749 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:46.692717 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:22:46.793029 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:22:46.792998 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:23:11.562463 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.562421 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-566ddbbf98-z4kql" podUID="70b71177-48b9-4fd8-a350-268a3e702f14" containerName="console" containerID="cri-o://8391cc954ba5f0a562cae3c6f7a7f257e492ab5f08b1516b29576705e873b686" gracePeriod=15 Jul 14 11:23:11.799564 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.799541 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:23:11.842888 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.842805 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-trusted-ca-bundle\") pod \"70b71177-48b9-4fd8-a350-268a3e702f14\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " Jul 14 11:23:11.842888 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.842848 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q74vh\" (UniqueName: \"kubernetes.io/projected/70b71177-48b9-4fd8-a350-268a3e702f14-kube-api-access-q74vh\") pod \"70b71177-48b9-4fd8-a350-268a3e702f14\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " Jul 14 11:23:11.843081 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.842890 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-service-ca\") pod \"70b71177-48b9-4fd8-a350-268a3e702f14\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " Jul 14 11:23:11.843081 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.842914 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70b71177-48b9-4fd8-a350-268a3e702f14-console-serving-cert\") pod \"70b71177-48b9-4fd8-a350-268a3e702f14\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " Jul 14 11:23:11.843081 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.842938 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70b71177-48b9-4fd8-a350-268a3e702f14-console-oauth-config\") pod \"70b71177-48b9-4fd8-a350-268a3e702f14\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " Jul 14 11:23:11.843081 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.843012 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-console-config\") pod \"70b71177-48b9-4fd8-a350-268a3e702f14\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " Jul 14 11:23:11.843081 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.843044 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-oauth-serving-cert\") pod \"70b71177-48b9-4fd8-a350-268a3e702f14\" (UID: \"70b71177-48b9-4fd8-a350-268a3e702f14\") " Jul 14 11:23:11.843360 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.843328 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-service-ca" (OuterVolumeSpecName: "service-ca") pod "70b71177-48b9-4fd8-a350-268a3e702f14" (UID: "70b71177-48b9-4fd8-a350-268a3e702f14"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:23:11.843560 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.843246 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "70b71177-48b9-4fd8-a350-268a3e702f14" (UID: "70b71177-48b9-4fd8-a350-268a3e702f14"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:23:11.843560 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.843475 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-console-config" (OuterVolumeSpecName: "console-config") pod "70b71177-48b9-4fd8-a350-268a3e702f14" (UID: "70b71177-48b9-4fd8-a350-268a3e702f14"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:23:11.843712 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.843575 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "70b71177-48b9-4fd8-a350-268a3e702f14" (UID: "70b71177-48b9-4fd8-a350-268a3e702f14"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:23:11.845206 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.845169 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70b71177-48b9-4fd8-a350-268a3e702f14-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "70b71177-48b9-4fd8-a350-268a3e702f14" (UID: "70b71177-48b9-4fd8-a350-268a3e702f14"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:23:11.845206 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.845187 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70b71177-48b9-4fd8-a350-268a3e702f14-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "70b71177-48b9-4fd8-a350-268a3e702f14" (UID: "70b71177-48b9-4fd8-a350-268a3e702f14"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:23:11.845331 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.845220 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70b71177-48b9-4fd8-a350-268a3e702f14-kube-api-access-q74vh" (OuterVolumeSpecName: "kube-api-access-q74vh") pod "70b71177-48b9-4fd8-a350-268a3e702f14" (UID: "70b71177-48b9-4fd8-a350-268a3e702f14"). InnerVolumeSpecName "kube-api-access-q74vh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:23:11.944039 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.943989 2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-service-ca\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:23:11.944039 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.944033 2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70b71177-48b9-4fd8-a350-268a3e702f14-console-serving-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:23:11.944039 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.944043 2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70b71177-48b9-4fd8-a350-268a3e702f14-console-oauth-config\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:23:11.944039 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.944054 2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-console-config\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:23:11.944311 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.944063 2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-oauth-serving-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:23:11.944311 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.944071 2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70b71177-48b9-4fd8-a350-268a3e702f14-trusted-ca-bundle\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:23:11.944311 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:11.944080 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q74vh\" (UniqueName: \"kubernetes.io/projected/70b71177-48b9-4fd8-a350-268a3e702f14-kube-api-access-q74vh\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:23:12.575049 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:12.575014 2571 generic.go:356] "Generic (PLEG): container finished" podID="70b71177-48b9-4fd8-a350-268a3e702f14" containerID="8391cc954ba5f0a562cae3c6f7a7f257e492ab5f08b1516b29576705e873b686" exitCode=0 Jul 14 11:23:12.575461 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:12.575057 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-566ddbbf98-z4kql" Jul 14 11:23:12.575461 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:12.575073 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-566ddbbf98-z4kql" event={"ID":"70b71177-48b9-4fd8-a350-268a3e702f14","Type":"ContainerDied","Data":"8391cc954ba5f0a562cae3c6f7a7f257e492ab5f08b1516b29576705e873b686"} Jul 14 11:23:12.575461 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:12.575112 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-566ddbbf98-z4kql" event={"ID":"70b71177-48b9-4fd8-a350-268a3e702f14","Type":"ContainerDied","Data":"f8c251ecf19bc93b03cb68aa9fad40a0d7eb06e9c351bc0f5513fe16341fde9c"} Jul 14 11:23:12.575461 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:12.575133 2571 scope.go:117] "RemoveContainer" containerID="8391cc954ba5f0a562cae3c6f7a7f257e492ab5f08b1516b29576705e873b686" Jul 14 11:23:12.583030 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:12.583006 2571 scope.go:117] "RemoveContainer" containerID="8391cc954ba5f0a562cae3c6f7a7f257e492ab5f08b1516b29576705e873b686" Jul 14 11:23:12.583304 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:23:12.583275 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8391cc954ba5f0a562cae3c6f7a7f257e492ab5f08b1516b29576705e873b686\": container with ID starting with 8391cc954ba5f0a562cae3c6f7a7f257e492ab5f08b1516b29576705e873b686 not found: ID does not exist" containerID="8391cc954ba5f0a562cae3c6f7a7f257e492ab5f08b1516b29576705e873b686" Jul 14 11:23:12.583354 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:12.583310 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8391cc954ba5f0a562cae3c6f7a7f257e492ab5f08b1516b29576705e873b686"} err="failed to get container status \"8391cc954ba5f0a562cae3c6f7a7f257e492ab5f08b1516b29576705e873b686\": rpc error: code = NotFound desc = could not find container \"8391cc954ba5f0a562cae3c6f7a7f257e492ab5f08b1516b29576705e873b686\": container with ID starting with 8391cc954ba5f0a562cae3c6f7a7f257e492ab5f08b1516b29576705e873b686 not found: ID does not exist" Jul 14 11:23:12.590970 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:12.590944 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-566ddbbf98-z4kql"] Jul 14 11:23:12.596764 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:12.596741 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-566ddbbf98-z4kql"] Jul 14 11:23:14.223130 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:14.223091 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:23:14.348650 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:14.348592 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:23:14.402386 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:14.402356 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70b71177-48b9-4fd8-a350-268a3e702f14" path="/var/lib/kubelet/pods/70b71177-48b9-4fd8-a350-268a3e702f14/volumes" Jul 14 11:23:15.112218 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:15.112184 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj"] Jul 14 11:23:15.112484 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:15.112473 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70b71177-48b9-4fd8-a350-268a3e702f14" containerName="console" Jul 14 11:23:15.112526 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:15.112485 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="70b71177-48b9-4fd8-a350-268a3e702f14" containerName="console" Jul 14 11:23:15.112563 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:15.112537 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="70b71177-48b9-4fd8-a350-268a3e702f14" containerName="console" Jul 14 11:23:15.116877 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:15.116855 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj" Jul 14 11:23:15.122206 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:15.122181 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 11:23:15.123586 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:15.123568 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4w4t9\"" Jul 14 11:23:15.123695 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:15.123591 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 11:23:15.131264 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:15.131239 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj"] Jul 14 11:23:15.167605 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:15.167575 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9412acec-ea69-4e8d-b71f-990c5f73bc09-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj\" (UID: \"9412acec-ea69-4e8d-b71f-990c5f73bc09\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj" Jul 14 11:23:15.167767 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:15.167644 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9412acec-ea69-4e8d-b71f-990c5f73bc09-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj\" (UID: \"9412acec-ea69-4e8d-b71f-990c5f73bc09\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj" Jul 14 11:23:15.167767 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:15.167683 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6c9k\" (UniqueName: \"kubernetes.io/projected/9412acec-ea69-4e8d-b71f-990c5f73bc09-kube-api-access-f6c9k\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj\" (UID: \"9412acec-ea69-4e8d-b71f-990c5f73bc09\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj" Jul 14 11:23:15.268216 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:15.268166 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6c9k\" (UniqueName: \"kubernetes.io/projected/9412acec-ea69-4e8d-b71f-990c5f73bc09-kube-api-access-f6c9k\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj\" (UID: \"9412acec-ea69-4e8d-b71f-990c5f73bc09\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj" Jul 14 11:23:15.268216 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:15.268232 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9412acec-ea69-4e8d-b71f-990c5f73bc09-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj\" (UID: \"9412acec-ea69-4e8d-b71f-990c5f73bc09\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj" Jul 14 11:23:15.268610 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:15.268277 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9412acec-ea69-4e8d-b71f-990c5f73bc09-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj\" (UID: \"9412acec-ea69-4e8d-b71f-990c5f73bc09\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj" Jul 14 11:23:15.268674 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:15.268616 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9412acec-ea69-4e8d-b71f-990c5f73bc09-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj\" (UID: \"9412acec-ea69-4e8d-b71f-990c5f73bc09\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj" Jul 14 11:23:15.268710 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:15.268681 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9412acec-ea69-4e8d-b71f-990c5f73bc09-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj\" (UID: \"9412acec-ea69-4e8d-b71f-990c5f73bc09\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj" Jul 14 11:23:15.278421 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:15.278397 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6c9k\" (UniqueName: \"kubernetes.io/projected/9412acec-ea69-4e8d-b71f-990c5f73bc09-kube-api-access-f6c9k\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj\" (UID: \"9412acec-ea69-4e8d-b71f-990c5f73bc09\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj" Jul 14 11:23:15.429819 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:15.429730 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj" Jul 14 11:23:15.565466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:15.565441 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj"] Jul 14 11:23:15.567982 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:23:15.567959 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9412acec_ea69_4e8d_b71f_990c5f73bc09.slice/crio-a0e61867f17d27acfe8771a3e50fbac1d72cdf5926a3be77e10ad2f10461648a WatchSource:0}: Error finding container a0e61867f17d27acfe8771a3e50fbac1d72cdf5926a3be77e10ad2f10461648a: Status 404 returned error can't find the container with id a0e61867f17d27acfe8771a3e50fbac1d72cdf5926a3be77e10ad2f10461648a Jul 14 11:23:15.588052 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:15.588019 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj" event={"ID":"9412acec-ea69-4e8d-b71f-990c5f73bc09","Type":"ContainerStarted","Data":"a0e61867f17d27acfe8771a3e50fbac1d72cdf5926a3be77e10ad2f10461648a"} Jul 14 11:23:22.619004 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:22.618969 2571 generic.go:356] "Generic (PLEG): container finished" podID="9412acec-ea69-4e8d-b71f-990c5f73bc09" containerID="9f1f62b1d7a0e3e53a04a62e4da730dab948d92dc9ecd141ee17bdf7f3f84b4d" exitCode=0 Jul 14 11:23:22.619373 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:22.619026 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj" event={"ID":"9412acec-ea69-4e8d-b71f-990c5f73bc09","Type":"ContainerDied","Data":"9f1f62b1d7a0e3e53a04a62e4da730dab948d92dc9ecd141ee17bdf7f3f84b4d"} Jul 14 11:23:25.633468 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:25.633431 2571 generic.go:356] "Generic (PLEG): container finished" podID="9412acec-ea69-4e8d-b71f-990c5f73bc09" containerID="2a2f823be7a1eb2d2a7d6a4b2f6dc092f5ccfc6267950582c3bb28525971ba27" exitCode=0 Jul 14 11:23:25.633468 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:25.633472 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj" event={"ID":"9412acec-ea69-4e8d-b71f-990c5f73bc09","Type":"ContainerDied","Data":"2a2f823be7a1eb2d2a7d6a4b2f6dc092f5ccfc6267950582c3bb28525971ba27"} Jul 14 11:23:26.266938 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:26.266909 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 11:23:26.268907 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:26.268888 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 11:23:26.270762 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:26.270746 2571 kubelet.go:1648] "Image garbage collection succeeded" Jul 14 11:23:33.668471 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:33.668435 2571 generic.go:356] "Generic (PLEG): container finished" podID="9412acec-ea69-4e8d-b71f-990c5f73bc09" containerID="6f3d00c74c192be8580894e4d24b57d4c89cdbb284f79679e5a5f54bacfcc1e5" exitCode=0 Jul 14 11:23:33.668912 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:33.668512 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj" event={"ID":"9412acec-ea69-4e8d-b71f-990c5f73bc09","Type":"ContainerDied","Data":"6f3d00c74c192be8580894e4d24b57d4c89cdbb284f79679e5a5f54bacfcc1e5"} Jul 14 11:23:34.800048 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:34.800024 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj" Jul 14 11:23:34.941344 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:34.941245 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9412acec-ea69-4e8d-b71f-990c5f73bc09-bundle\") pod \"9412acec-ea69-4e8d-b71f-990c5f73bc09\" (UID: \"9412acec-ea69-4e8d-b71f-990c5f73bc09\") " Jul 14 11:23:34.941344 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:34.941325 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6c9k\" (UniqueName: \"kubernetes.io/projected/9412acec-ea69-4e8d-b71f-990c5f73bc09-kube-api-access-f6c9k\") pod \"9412acec-ea69-4e8d-b71f-990c5f73bc09\" (UID: \"9412acec-ea69-4e8d-b71f-990c5f73bc09\") " Jul 14 11:23:34.941573 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:34.941373 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9412acec-ea69-4e8d-b71f-990c5f73bc09-util\") pod \"9412acec-ea69-4e8d-b71f-990c5f73bc09\" (UID: \"9412acec-ea69-4e8d-b71f-990c5f73bc09\") " Jul 14 11:23:34.941973 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:34.941934 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9412acec-ea69-4e8d-b71f-990c5f73bc09-bundle" (OuterVolumeSpecName: "bundle") pod "9412acec-ea69-4e8d-b71f-990c5f73bc09" (UID: "9412acec-ea69-4e8d-b71f-990c5f73bc09"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:23:34.943524 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:34.943497 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9412acec-ea69-4e8d-b71f-990c5f73bc09-kube-api-access-f6c9k" (OuterVolumeSpecName: "kube-api-access-f6c9k") pod "9412acec-ea69-4e8d-b71f-990c5f73bc09" (UID: "9412acec-ea69-4e8d-b71f-990c5f73bc09"). InnerVolumeSpecName "kube-api-access-f6c9k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:23:34.946251 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:34.946224 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9412acec-ea69-4e8d-b71f-990c5f73bc09-util" (OuterVolumeSpecName: "util") pod "9412acec-ea69-4e8d-b71f-990c5f73bc09" (UID: "9412acec-ea69-4e8d-b71f-990c5f73bc09"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:23:35.042059 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:35.042021 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6c9k\" (UniqueName: \"kubernetes.io/projected/9412acec-ea69-4e8d-b71f-990c5f73bc09-kube-api-access-f6c9k\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:23:35.042059 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:35.042054 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9412acec-ea69-4e8d-b71f-990c5f73bc09-util\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:23:35.042059 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:35.042063 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9412acec-ea69-4e8d-b71f-990c5f73bc09-bundle\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:23:35.678385 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:35.678355 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj" Jul 14 11:23:35.678625 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:35.678355 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675405dpnj" event={"ID":"9412acec-ea69-4e8d-b71f-990c5f73bc09","Type":"ContainerDied","Data":"a0e61867f17d27acfe8771a3e50fbac1d72cdf5926a3be77e10ad2f10461648a"} Jul 14 11:23:35.678625 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:35.678473 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0e61867f17d27acfe8771a3e50fbac1d72cdf5926a3be77e10ad2f10461648a" Jul 14 11:23:41.165925 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:41.165883 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:23:42.757322 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:42.757284 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:23:43.140343 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.140309 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jx7bx"] Jul 14 11:23:43.140586 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.140575 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9412acec-ea69-4e8d-b71f-990c5f73bc09" containerName="extract" Jul 14 11:23:43.140649 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.140587 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9412acec-ea69-4e8d-b71f-990c5f73bc09" containerName="extract" Jul 14 11:23:43.140649 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.140601 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9412acec-ea69-4e8d-b71f-990c5f73bc09" containerName="util" Jul 14 11:23:43.140649 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.140606 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9412acec-ea69-4e8d-b71f-990c5f73bc09" containerName="util" Jul 14 11:23:43.140752 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.140648 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9412acec-ea69-4e8d-b71f-990c5f73bc09" containerName="pull" Jul 14 11:23:43.140752 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.140658 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9412acec-ea69-4e8d-b71f-990c5f73bc09" containerName="pull" Jul 14 11:23:43.140752 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.140714 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9412acec-ea69-4e8d-b71f-990c5f73bc09" containerName="extract" Jul 14 11:23:43.185898 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.185866 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jx7bx"] Jul 14 11:23:43.186044 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.185965 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jx7bx" Jul 14 11:23:43.189545 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.189525 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 14 11:23:43.190466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.190447 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 14 11:23:43.190539 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.190494 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-8vzs5\"" Jul 14 11:23:43.309471 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.309439 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7hm5\" (UniqueName: \"kubernetes.io/projected/00c14117-67fc-4bf0-8da2-db96b99ac99c-kube-api-access-t7hm5\") pod \"cert-manager-operator-controller-manager-55c444bf86-jx7bx\" (UID: \"00c14117-67fc-4bf0-8da2-db96b99ac99c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jx7bx" Jul 14 11:23:43.309625 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.309487 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00c14117-67fc-4bf0-8da2-db96b99ac99c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jx7bx\" (UID: \"00c14117-67fc-4bf0-8da2-db96b99ac99c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jx7bx" Jul 14 11:23:43.410754 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.410684 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7hm5\" (UniqueName: \"kubernetes.io/projected/00c14117-67fc-4bf0-8da2-db96b99ac99c-kube-api-access-t7hm5\") pod \"cert-manager-operator-controller-manager-55c444bf86-jx7bx\" (UID: \"00c14117-67fc-4bf0-8da2-db96b99ac99c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jx7bx" Jul 14 11:23:43.410754 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.410729 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00c14117-67fc-4bf0-8da2-db96b99ac99c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jx7bx\" (UID: \"00c14117-67fc-4bf0-8da2-db96b99ac99c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jx7bx" Jul 14 11:23:43.411077 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.411062 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00c14117-67fc-4bf0-8da2-db96b99ac99c-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jx7bx\" (UID: \"00c14117-67fc-4bf0-8da2-db96b99ac99c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jx7bx" Jul 14 11:23:43.421194 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.421169 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7hm5\" (UniqueName: \"kubernetes.io/projected/00c14117-67fc-4bf0-8da2-db96b99ac99c-kube-api-access-t7hm5\") pod \"cert-manager-operator-controller-manager-55c444bf86-jx7bx\" (UID: \"00c14117-67fc-4bf0-8da2-db96b99ac99c\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jx7bx" Jul 14 11:23:43.494484 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.494456 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jx7bx" Jul 14 11:23:43.637907 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.637887 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jx7bx"] Jul 14 11:23:43.640656 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:23:43.640611 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00c14117_67fc_4bf0_8da2_db96b99ac99c.slice/crio-79323045cdd4228cf20805872aadf83c31b9ab89248caba5950f5f8b86b2cbe1 WatchSource:0}: Error finding container 79323045cdd4228cf20805872aadf83c31b9ab89248caba5950f5f8b86b2cbe1: Status 404 returned error can't find the container with id 79323045cdd4228cf20805872aadf83c31b9ab89248caba5950f5f8b86b2cbe1 Jul 14 11:23:43.643138 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.643121 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 11:23:43.708817 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:43.708756 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jx7bx" event={"ID":"00c14117-67fc-4bf0-8da2-db96b99ac99c","Type":"ContainerStarted","Data":"79323045cdd4228cf20805872aadf83c31b9ab89248caba5950f5f8b86b2cbe1"} Jul 14 11:23:44.779814 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:44.779778 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:23:44.880315 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:44.880275 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:23:45.718827 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:45.718791 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jx7bx" event={"ID":"00c14117-67fc-4bf0-8da2-db96b99ac99c","Type":"ContainerStarted","Data":"29051dddd0d6373a6c3d1c3d9ea3e39aaf145eaac5322ccc0f1956c36a8efa3a"} Jul 14 11:23:45.745650 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:45.745163 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jx7bx" podStartSLOduration=1.048159165 podStartE2EDuration="2.745142411s" totalImagesPullingTime="1.696983246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:23:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:23:45.741450966 +0000 UTC m=+319.902527434" watchObservedRunningTime="2026-07-14 11:23:45.745142411 +0000 UTC m=+319.906218884" Jul 14 11:23:46.142130 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:46.142099 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:23:46.278342 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:46.278308 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:23:46.811760 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:46.811733 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm"] Jul 14 11:23:46.814180 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:46.814163 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm" Jul 14 11:23:46.817387 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:46.817366 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 11:23:46.817479 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:46.817369 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4w4t9\"" Jul 14 11:23:46.818669 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:46.818655 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 11:23:46.836867 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:46.836845 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm"] Jul 14 11:23:46.943510 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:46.943476 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsjzq\" (UniqueName: \"kubernetes.io/projected/a599df1a-01db-46a3-98a5-0c55d3c92aa6-kube-api-access-xsjzq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm\" (UID: \"a599df1a-01db-46a3-98a5-0c55d3c92aa6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm" Jul 14 11:23:46.943699 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:46.943529 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a599df1a-01db-46a3-98a5-0c55d3c92aa6-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm\" (UID: \"a599df1a-01db-46a3-98a5-0c55d3c92aa6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm" Jul 14 11:23:46.943699 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:46.943560 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a599df1a-01db-46a3-98a5-0c55d3c92aa6-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm\" (UID: \"a599df1a-01db-46a3-98a5-0c55d3c92aa6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm" Jul 14 11:23:47.044835 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.044802 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a599df1a-01db-46a3-98a5-0c55d3c92aa6-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm\" (UID: \"a599df1a-01db-46a3-98a5-0c55d3c92aa6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm" Jul 14 11:23:47.044992 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.044842 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a599df1a-01db-46a3-98a5-0c55d3c92aa6-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm\" (UID: \"a599df1a-01db-46a3-98a5-0c55d3c92aa6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm" Jul 14 11:23:47.044992 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.044886 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsjzq\" (UniqueName: \"kubernetes.io/projected/a599df1a-01db-46a3-98a5-0c55d3c92aa6-kube-api-access-xsjzq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm\" (UID: \"a599df1a-01db-46a3-98a5-0c55d3c92aa6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm" Jul 14 11:23:47.045187 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.045168 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a599df1a-01db-46a3-98a5-0c55d3c92aa6-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm\" (UID: \"a599df1a-01db-46a3-98a5-0c55d3c92aa6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm" Jul 14 11:23:47.045250 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.045211 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a599df1a-01db-46a3-98a5-0c55d3c92aa6-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm\" (UID: \"a599df1a-01db-46a3-98a5-0c55d3c92aa6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm" Jul 14 11:23:47.060582 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.060555 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsjzq\" (UniqueName: \"kubernetes.io/projected/a599df1a-01db-46a3-98a5-0c55d3c92aa6-kube-api-access-xsjzq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm\" (UID: \"a599df1a-01db-46a3-98a5-0c55d3c92aa6\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm" Jul 14 11:23:47.083982 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.083918 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:23:47.122901 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.122872 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm" Jul 14 11:23:47.184657 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.184575 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:23:47.192147 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.192120 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4mx2x"] Jul 14 11:23:47.195128 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.195108 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-4mx2x" Jul 14 11:23:47.198142 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.198122 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 14 11:23:47.198248 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.198182 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 14 11:23:47.198313 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.198279 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-bqb56\"" Jul 14 11:23:47.205812 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.205778 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4mx2x"] Jul 14 11:23:47.255799 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.255779 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm"] Jul 14 11:23:47.257559 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:23:47.257533 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda599df1a_01db_46a3_98a5_0c55d3c92aa6.slice/crio-1d9648b353caebc9a60a44951f78ecc7183e8c5305e37040be84c7d646bd644c WatchSource:0}: Error finding container 1d9648b353caebc9a60a44951f78ecc7183e8c5305e37040be84c7d646bd644c: Status 404 returned error can't find the container with id 1d9648b353caebc9a60a44951f78ecc7183e8c5305e37040be84c7d646bd644c Jul 14 11:23:47.347193 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.347167 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9pqz\" (UniqueName: \"kubernetes.io/projected/16964359-f17a-4324-8904-96acff496a5c-kube-api-access-r9pqz\") pod \"cert-manager-webhook-6486ffd997-4mx2x\" (UID: \"16964359-f17a-4324-8904-96acff496a5c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4mx2x" Jul 14 11:23:47.347372 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.347221 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16964359-f17a-4324-8904-96acff496a5c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4mx2x\" (UID: \"16964359-f17a-4324-8904-96acff496a5c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4mx2x" Jul 14 11:23:47.447941 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.447910 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9pqz\" (UniqueName: \"kubernetes.io/projected/16964359-f17a-4324-8904-96acff496a5c-kube-api-access-r9pqz\") pod \"cert-manager-webhook-6486ffd997-4mx2x\" (UID: \"16964359-f17a-4324-8904-96acff496a5c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4mx2x" Jul 14 11:23:47.448100 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.447964 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16964359-f17a-4324-8904-96acff496a5c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4mx2x\" (UID: \"16964359-f17a-4324-8904-96acff496a5c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4mx2x" Jul 14 11:23:47.461720 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.461690 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/16964359-f17a-4324-8904-96acff496a5c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4mx2x\" (UID: \"16964359-f17a-4324-8904-96acff496a5c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4mx2x" Jul 14 11:23:47.461908 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.461891 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9pqz\" (UniqueName: \"kubernetes.io/projected/16964359-f17a-4324-8904-96acff496a5c-kube-api-access-r9pqz\") pod \"cert-manager-webhook-6486ffd997-4mx2x\" (UID: \"16964359-f17a-4324-8904-96acff496a5c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4mx2x" Jul 14 11:23:47.520019 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.519992 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-4mx2x" Jul 14 11:23:47.647557 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.647505 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4mx2x"] Jul 14 11:23:47.650033 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:23:47.649998 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16964359_f17a_4324_8904_96acff496a5c.slice/crio-9ce9c90a5d07719f56c029b6bb7ffab735e6f21ed13c051ff7c4e72271bef2b2 WatchSource:0}: Error finding container 9ce9c90a5d07719f56c029b6bb7ffab735e6f21ed13c051ff7c4e72271bef2b2: Status 404 returned error can't find the container with id 9ce9c90a5d07719f56c029b6bb7ffab735e6f21ed13c051ff7c4e72271bef2b2 Jul 14 11:23:47.728203 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.728162 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-4mx2x" event={"ID":"16964359-f17a-4324-8904-96acff496a5c","Type":"ContainerStarted","Data":"9ce9c90a5d07719f56c029b6bb7ffab735e6f21ed13c051ff7c4e72271bef2b2"} Jul 14 11:23:47.729570 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.729548 2571 generic.go:356] "Generic (PLEG): container finished" podID="a599df1a-01db-46a3-98a5-0c55d3c92aa6" containerID="c6db66a039ea9bd69cfde4ad45a2d6e3281402877be0157b792a027e3a530825" exitCode=0 Jul 14 11:23:47.729702 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.729595 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm" event={"ID":"a599df1a-01db-46a3-98a5-0c55d3c92aa6","Type":"ContainerDied","Data":"c6db66a039ea9bd69cfde4ad45a2d6e3281402877be0157b792a027e3a530825"} Jul 14 11:23:47.729702 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:47.729655 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm" event={"ID":"a599df1a-01db-46a3-98a5-0c55d3c92aa6","Type":"ContainerStarted","Data":"1d9648b353caebc9a60a44951f78ecc7183e8c5305e37040be84c7d646bd644c"} Jul 14 11:23:49.251081 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:49.251009 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:23:49.678688 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:49.678649 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:23:49.737286 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:49.737246 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-2fszn"] Jul 14 11:23:49.740075 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:49.740055 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2fszn" Jul 14 11:23:49.743021 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:49.742998 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-p9p8j\"" Jul 14 11:23:49.751069 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:49.751045 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-2fszn"] Jul 14 11:23:49.873586 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:49.873553 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/695fbdf2-9e3b-4c04-af21-6203ae0fa1f7-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-2fszn\" (UID: \"695fbdf2-9e3b-4c04-af21-6203ae0fa1f7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2fszn" Jul 14 11:23:49.873770 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:49.873604 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48vtn\" (UniqueName: \"kubernetes.io/projected/695fbdf2-9e3b-4c04-af21-6203ae0fa1f7-kube-api-access-48vtn\") pod \"cert-manager-cainjector-559cdd8c87-2fszn\" (UID: \"695fbdf2-9e3b-4c04-af21-6203ae0fa1f7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2fszn" Jul 14 11:23:49.975013 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:49.974922 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/695fbdf2-9e3b-4c04-af21-6203ae0fa1f7-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-2fszn\" (UID: \"695fbdf2-9e3b-4c04-af21-6203ae0fa1f7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2fszn" Jul 14 11:23:49.975013 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:49.975002 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48vtn\" (UniqueName: \"kubernetes.io/projected/695fbdf2-9e3b-4c04-af21-6203ae0fa1f7-kube-api-access-48vtn\") pod \"cert-manager-cainjector-559cdd8c87-2fszn\" (UID: \"695fbdf2-9e3b-4c04-af21-6203ae0fa1f7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2fszn" Jul 14 11:23:49.984491 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:49.984442 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/695fbdf2-9e3b-4c04-af21-6203ae0fa1f7-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-2fszn\" (UID: \"695fbdf2-9e3b-4c04-af21-6203ae0fa1f7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2fszn" Jul 14 11:23:49.984653 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:49.984546 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48vtn\" (UniqueName: \"kubernetes.io/projected/695fbdf2-9e3b-4c04-af21-6203ae0fa1f7-kube-api-access-48vtn\") pod \"cert-manager-cainjector-559cdd8c87-2fszn\" (UID: \"695fbdf2-9e3b-4c04-af21-6203ae0fa1f7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2fszn" Jul 14 11:23:50.050774 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:50.050740 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2fszn" Jul 14 11:23:51.120920 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:51.120731 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-2fszn"] Jul 14 11:23:51.123763 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:23:51.123723 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod695fbdf2_9e3b_4c04_af21_6203ae0fa1f7.slice/crio-a33a662134566ec4fc9ef528d7805ed0caf47ec27e58aa378d6741724a99da7b WatchSource:0}: Error finding container a33a662134566ec4fc9ef528d7805ed0caf47ec27e58aa378d6741724a99da7b: Status 404 returned error can't find the container with id a33a662134566ec4fc9ef528d7805ed0caf47ec27e58aa378d6741724a99da7b Jul 14 11:23:51.569780 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:51.569738 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:23:51.631205 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:51.631163 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:23:51.749872 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:51.749837 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-4mx2x" event={"ID":"16964359-f17a-4324-8904-96acff496a5c","Type":"ContainerStarted","Data":"b194165f295763df9e86f07bb7c0c8c9624293aa14c657493844aa2766da8558"} Jul 14 11:23:51.750049 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:51.749908 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-4mx2x" Jul 14 11:23:51.751370 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:51.751343 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2fszn" event={"ID":"695fbdf2-9e3b-4c04-af21-6203ae0fa1f7","Type":"ContainerStarted","Data":"1f85f8b69445bcef3a1acf5c38d7358548a84b6719b4669ad700f30d20605422"} Jul 14 11:23:51.751482 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:51.751380 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2fszn" event={"ID":"695fbdf2-9e3b-4c04-af21-6203ae0fa1f7","Type":"ContainerStarted","Data":"a33a662134566ec4fc9ef528d7805ed0caf47ec27e58aa378d6741724a99da7b"} Jul 14 11:23:51.752933 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:51.752911 2571 generic.go:356] "Generic (PLEG): container finished" podID="a599df1a-01db-46a3-98a5-0c55d3c92aa6" containerID="38f7984bf9ae4a845a34ff6467d7f1bc359c0c5a42a87507933b67ae075ec563" exitCode=0 Jul 14 11:23:51.753042 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:51.752947 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm" event={"ID":"a599df1a-01db-46a3-98a5-0c55d3c92aa6","Type":"ContainerDied","Data":"38f7984bf9ae4a845a34ff6467d7f1bc359c0c5a42a87507933b67ae075ec563"} Jul 14 11:23:51.767278 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:51.767237 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-4mx2x" podStartSLOduration=1.431877573 podStartE2EDuration="4.767223305s" totalImagesPullingTime="3.335345732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:23:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:23:51.765514305 +0000 UTC m=+325.926590784" watchObservedRunningTime="2026-07-14 11:23:51.767223305 +0000 UTC m=+325.928299772" Jul 14 11:23:51.836773 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:51.836684 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2fszn" podStartSLOduration=2.836668351 podStartE2EDuration="2.836668351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:23:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:23:51.834921371 +0000 UTC m=+325.995997851" watchObservedRunningTime="2026-07-14 11:23:51.836668351 +0000 UTC m=+325.997744818" Jul 14 11:23:52.759535 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:52.759498 2571 generic.go:356] "Generic (PLEG): container finished" podID="a599df1a-01db-46a3-98a5-0c55d3c92aa6" containerID="62d60c3728958f2d7072e07f9c05f988b4aea5b0a61b75062ccafac0cc529dbb" exitCode=0 Jul 14 11:23:52.759977 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:52.759560 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm" event={"ID":"a599df1a-01db-46a3-98a5-0c55d3c92aa6","Type":"ContainerDied","Data":"62d60c3728958f2d7072e07f9c05f988b4aea5b0a61b75062ccafac0cc529dbb"} Jul 14 11:23:53.893497 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:53.893469 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm" Jul 14 11:23:53.909471 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:53.909448 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a599df1a-01db-46a3-98a5-0c55d3c92aa6-util\") pod \"a599df1a-01db-46a3-98a5-0c55d3c92aa6\" (UID: \"a599df1a-01db-46a3-98a5-0c55d3c92aa6\") " Jul 14 11:23:53.909603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:53.909480 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsjzq\" (UniqueName: \"kubernetes.io/projected/a599df1a-01db-46a3-98a5-0c55d3c92aa6-kube-api-access-xsjzq\") pod \"a599df1a-01db-46a3-98a5-0c55d3c92aa6\" (UID: \"a599df1a-01db-46a3-98a5-0c55d3c92aa6\") " Jul 14 11:23:53.909603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:53.909508 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a599df1a-01db-46a3-98a5-0c55d3c92aa6-bundle\") pod \"a599df1a-01db-46a3-98a5-0c55d3c92aa6\" (UID: \"a599df1a-01db-46a3-98a5-0c55d3c92aa6\") " Jul 14 11:23:53.909939 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:53.909914 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a599df1a-01db-46a3-98a5-0c55d3c92aa6-bundle" (OuterVolumeSpecName: "bundle") pod "a599df1a-01db-46a3-98a5-0c55d3c92aa6" (UID: "a599df1a-01db-46a3-98a5-0c55d3c92aa6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:23:53.911664 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:53.911604 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a599df1a-01db-46a3-98a5-0c55d3c92aa6-kube-api-access-xsjzq" (OuterVolumeSpecName: "kube-api-access-xsjzq") pod "a599df1a-01db-46a3-98a5-0c55d3c92aa6" (UID: "a599df1a-01db-46a3-98a5-0c55d3c92aa6"). InnerVolumeSpecName "kube-api-access-xsjzq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:23:53.914134 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:53.914112 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a599df1a-01db-46a3-98a5-0c55d3c92aa6-util" (OuterVolumeSpecName: "util") pod "a599df1a-01db-46a3-98a5-0c55d3c92aa6" (UID: "a599df1a-01db-46a3-98a5-0c55d3c92aa6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:23:54.010224 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:54.010182 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a599df1a-01db-46a3-98a5-0c55d3c92aa6-util\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:23:54.010224 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:54.010216 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsjzq\" (UniqueName: \"kubernetes.io/projected/a599df1a-01db-46a3-98a5-0c55d3c92aa6-kube-api-access-xsjzq\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:23:54.010224 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:54.010225 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a599df1a-01db-46a3-98a5-0c55d3c92aa6-bundle\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:23:54.769332 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:54.769255 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm" Jul 14 11:23:54.769332 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:54.769268 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frgvkm" event={"ID":"a599df1a-01db-46a3-98a5-0c55d3c92aa6","Type":"ContainerDied","Data":"1d9648b353caebc9a60a44951f78ecc7183e8c5305e37040be84c7d646bd644c"} Jul 14 11:23:54.769332 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:54.769310 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d9648b353caebc9a60a44951f78ecc7183e8c5305e37040be84c7d646bd644c" Jul 14 11:23:57.266170 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:57.266143 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:23:57.761983 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:57.761955 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-4mx2x" Jul 14 11:23:59.471054 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:59.471019 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:23:59.572068 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:23:59.572038 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:00.049461 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.049428 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lxrlf"] Jul 14 11:24:00.049763 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.049749 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a599df1a-01db-46a3-98a5-0c55d3c92aa6" containerName="util" Jul 14 11:24:00.049826 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.049764 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a599df1a-01db-46a3-98a5-0c55d3c92aa6" containerName="util" Jul 14 11:24:00.049826 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.049774 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a599df1a-01db-46a3-98a5-0c55d3c92aa6" containerName="extract" Jul 14 11:24:00.049826 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.049779 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a599df1a-01db-46a3-98a5-0c55d3c92aa6" containerName="extract" Jul 14 11:24:00.049826 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.049787 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a599df1a-01db-46a3-98a5-0c55d3c92aa6" containerName="pull" Jul 14 11:24:00.049826 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.049792 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a599df1a-01db-46a3-98a5-0c55d3c92aa6" containerName="pull" Jul 14 11:24:00.049984 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.049846 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a599df1a-01db-46a3-98a5-0c55d3c92aa6" containerName="extract" Jul 14 11:24:00.051856 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.051838 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lxrlf" Jul 14 11:24:00.054668 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.054643 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-vsr6n\"" Jul 14 11:24:00.063004 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.062982 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lxrlf"] Jul 14 11:24:00.155168 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.155129 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5397ec09-5522-4da3-9e66-a0d5aa3af0b5-bound-sa-token\") pod \"cert-manager-55b4d954d5-lxrlf\" (UID: \"5397ec09-5522-4da3-9e66-a0d5aa3af0b5\") " pod="cert-manager/cert-manager-55b4d954d5-lxrlf" Jul 14 11:24:00.155339 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.155244 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldsnw\" (UniqueName: \"kubernetes.io/projected/5397ec09-5522-4da3-9e66-a0d5aa3af0b5-kube-api-access-ldsnw\") pod \"cert-manager-55b4d954d5-lxrlf\" (UID: \"5397ec09-5522-4da3-9e66-a0d5aa3af0b5\") " pod="cert-manager/cert-manager-55b4d954d5-lxrlf" Jul 14 11:24:00.256514 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.256475 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldsnw\" (UniqueName: \"kubernetes.io/projected/5397ec09-5522-4da3-9e66-a0d5aa3af0b5-kube-api-access-ldsnw\") pod \"cert-manager-55b4d954d5-lxrlf\" (UID: \"5397ec09-5522-4da3-9e66-a0d5aa3af0b5\") " pod="cert-manager/cert-manager-55b4d954d5-lxrlf" Jul 14 11:24:00.256702 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.256539 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5397ec09-5522-4da3-9e66-a0d5aa3af0b5-bound-sa-token\") pod \"cert-manager-55b4d954d5-lxrlf\" (UID: \"5397ec09-5522-4da3-9e66-a0d5aa3af0b5\") " pod="cert-manager/cert-manager-55b4d954d5-lxrlf" Jul 14 11:24:00.266779 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.266757 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5397ec09-5522-4da3-9e66-a0d5aa3af0b5-bound-sa-token\") pod \"cert-manager-55b4d954d5-lxrlf\" (UID: \"5397ec09-5522-4da3-9e66-a0d5aa3af0b5\") " pod="cert-manager/cert-manager-55b4d954d5-lxrlf" Jul 14 11:24:00.266905 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.266884 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldsnw\" (UniqueName: \"kubernetes.io/projected/5397ec09-5522-4da3-9e66-a0d5aa3af0b5-kube-api-access-ldsnw\") pod \"cert-manager-55b4d954d5-lxrlf\" (UID: \"5397ec09-5522-4da3-9e66-a0d5aa3af0b5\") " pod="cert-manager/cert-manager-55b4d954d5-lxrlf" Jul 14 11:24:00.360743 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.360716 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lxrlf" Jul 14 11:24:00.486899 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.486865 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lxrlf"] Jul 14 11:24:00.489810 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:24:00.489776 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5397ec09_5522_4da3_9e66_a0d5aa3af0b5.slice/crio-9eec281fcb73e3fb8848eb03e852149ba2a3db3032b0bfd8a222b1b6472734f1 WatchSource:0}: Error finding container 9eec281fcb73e3fb8848eb03e852149ba2a3db3032b0bfd8a222b1b6472734f1: Status 404 returned error can't find the container with id 9eec281fcb73e3fb8848eb03e852149ba2a3db3032b0bfd8a222b1b6472734f1 Jul 14 11:24:00.796395 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.796314 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lxrlf" event={"ID":"5397ec09-5522-4da3-9e66-a0d5aa3af0b5","Type":"ContainerStarted","Data":"0847cc27585e704a0fc5be2e365966ab5fc10367db66c9b75936553acef4baed"} Jul 14 11:24:00.796395 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.796360 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lxrlf" event={"ID":"5397ec09-5522-4da3-9e66-a0d5aa3af0b5","Type":"ContainerStarted","Data":"9eec281fcb73e3fb8848eb03e852149ba2a3db3032b0bfd8a222b1b6472734f1"} Jul 14 11:24:00.812004 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:00.811952 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-lxrlf" podStartSLOduration=0.811934382 podStartE2EDuration="811.934382ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:24:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:24:00.811667014 +0000 UTC m=+334.972743482" watchObservedRunningTime="2026-07-14 11:24:00.811934382 +0000 UTC m=+334.973010849" Jul 14 11:24:05.939783 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:05.939752 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:06.040651 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:06.040603 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:07.534149 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:07.534118 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp"] Jul 14 11:24:07.537802 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:07.537786 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp" Jul 14 11:24:07.540858 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:07.540831 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 11:24:07.542068 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:07.542045 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4w4t9\"" Jul 14 11:24:07.542128 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:07.542089 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 11:24:07.547053 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:07.547029 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp"] Jul 14 11:24:07.617943 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:07.617909 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/859f7201-3171-4fe2-aeec-72bdf8fef083-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp\" (UID: \"859f7201-3171-4fe2-aeec-72bdf8fef083\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp" Jul 14 11:24:07.618094 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:07.617961 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85pd8\" (UniqueName: \"kubernetes.io/projected/859f7201-3171-4fe2-aeec-72bdf8fef083-kube-api-access-85pd8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp\" (UID: \"859f7201-3171-4fe2-aeec-72bdf8fef083\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp" Jul 14 11:24:07.618094 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:07.618022 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/859f7201-3171-4fe2-aeec-72bdf8fef083-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp\" (UID: \"859f7201-3171-4fe2-aeec-72bdf8fef083\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp" Jul 14 11:24:07.719044 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:07.719004 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85pd8\" (UniqueName: \"kubernetes.io/projected/859f7201-3171-4fe2-aeec-72bdf8fef083-kube-api-access-85pd8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp\" (UID: \"859f7201-3171-4fe2-aeec-72bdf8fef083\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp" Jul 14 11:24:07.719204 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:07.719056 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/859f7201-3171-4fe2-aeec-72bdf8fef083-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp\" (UID: \"859f7201-3171-4fe2-aeec-72bdf8fef083\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp" Jul 14 11:24:07.719204 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:07.719105 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/859f7201-3171-4fe2-aeec-72bdf8fef083-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp\" (UID: \"859f7201-3171-4fe2-aeec-72bdf8fef083\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp" Jul 14 11:24:07.719521 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:07.719498 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/859f7201-3171-4fe2-aeec-72bdf8fef083-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp\" (UID: \"859f7201-3171-4fe2-aeec-72bdf8fef083\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp" Jul 14 11:24:07.719521 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:07.719513 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/859f7201-3171-4fe2-aeec-72bdf8fef083-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp\" (UID: \"859f7201-3171-4fe2-aeec-72bdf8fef083\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp" Jul 14 11:24:07.729167 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:07.729132 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85pd8\" (UniqueName: \"kubernetes.io/projected/859f7201-3171-4fe2-aeec-72bdf8fef083-kube-api-access-85pd8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp\" (UID: \"859f7201-3171-4fe2-aeec-72bdf8fef083\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp" Jul 14 11:24:07.847145 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:07.847115 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:07.848013 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:07.847996 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp" Jul 14 11:24:07.993609 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:07.993585 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp"] Jul 14 11:24:07.995666 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:24:07.995622 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod859f7201_3171_4fe2_aeec_72bdf8fef083.slice/crio-d632a02d29951a982fb29b6c8586adc50d3d9a4677644e996d8a113c21df072f WatchSource:0}: Error finding container d632a02d29951a982fb29b6c8586adc50d3d9a4677644e996d8a113c21df072f: Status 404 returned error can't find the container with id d632a02d29951a982fb29b6c8586adc50d3d9a4677644e996d8a113c21df072f Jul 14 11:24:08.831623 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:08.831591 2571 generic.go:356] "Generic (PLEG): container finished" podID="859f7201-3171-4fe2-aeec-72bdf8fef083" containerID="86058023397ebd8b0103035d8f1e8391c4bedbae8a9cc94d495c108d1e7aa83e" exitCode=0 Jul 14 11:24:08.831992 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:08.831679 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp" event={"ID":"859f7201-3171-4fe2-aeec-72bdf8fef083","Type":"ContainerDied","Data":"86058023397ebd8b0103035d8f1e8391c4bedbae8a9cc94d495c108d1e7aa83e"} Jul 14 11:24:08.831992 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:08.831728 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp" event={"ID":"859f7201-3171-4fe2-aeec-72bdf8fef083","Type":"ContainerStarted","Data":"d632a02d29951a982fb29b6c8586adc50d3d9a4677644e996d8a113c21df072f"} Jul 14 11:24:09.838292 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:09.838206 2571 generic.go:356] "Generic (PLEG): container finished" podID="859f7201-3171-4fe2-aeec-72bdf8fef083" containerID="8878cfbe2687535921cf166bec797b73396abd9bc3b2dba849a476fbf5bd8b0c" exitCode=0 Jul 14 11:24:09.838659 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:09.838291 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp" event={"ID":"859f7201-3171-4fe2-aeec-72bdf8fef083","Type":"ContainerDied","Data":"8878cfbe2687535921cf166bec797b73396abd9bc3b2dba849a476fbf5bd8b0c"} Jul 14 11:24:10.352281 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:10.352240 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:10.437804 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:10.437770 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:10.846342 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:10.846310 2571 generic.go:356] "Generic (PLEG): container finished" podID="859f7201-3171-4fe2-aeec-72bdf8fef083" containerID="134e1883284c4e9b57abefc6c783ee3e202882c74029eb94842c998d4c5dbee6" exitCode=0 Jul 14 11:24:10.846693 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:10.846368 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp" event={"ID":"859f7201-3171-4fe2-aeec-72bdf8fef083","Type":"ContainerDied","Data":"134e1883284c4e9b57abefc6c783ee3e202882c74029eb94842c998d4c5dbee6"} Jul 14 11:24:11.972081 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:11.972057 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp" Jul 14 11:24:12.051526 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:12.051491 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/859f7201-3171-4fe2-aeec-72bdf8fef083-bundle\") pod \"859f7201-3171-4fe2-aeec-72bdf8fef083\" (UID: \"859f7201-3171-4fe2-aeec-72bdf8fef083\") " Jul 14 11:24:12.051526 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:12.051532 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/859f7201-3171-4fe2-aeec-72bdf8fef083-util\") pod \"859f7201-3171-4fe2-aeec-72bdf8fef083\" (UID: \"859f7201-3171-4fe2-aeec-72bdf8fef083\") " Jul 14 11:24:12.051725 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:12.051558 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85pd8\" (UniqueName: \"kubernetes.io/projected/859f7201-3171-4fe2-aeec-72bdf8fef083-kube-api-access-85pd8\") pod \"859f7201-3171-4fe2-aeec-72bdf8fef083\" (UID: \"859f7201-3171-4fe2-aeec-72bdf8fef083\") " Jul 14 11:24:12.052414 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:12.052380 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/859f7201-3171-4fe2-aeec-72bdf8fef083-bundle" (OuterVolumeSpecName: "bundle") pod "859f7201-3171-4fe2-aeec-72bdf8fef083" (UID: "859f7201-3171-4fe2-aeec-72bdf8fef083"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:24:12.053583 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:12.053555 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/859f7201-3171-4fe2-aeec-72bdf8fef083-kube-api-access-85pd8" (OuterVolumeSpecName: "kube-api-access-85pd8") pod "859f7201-3171-4fe2-aeec-72bdf8fef083" (UID: "859f7201-3171-4fe2-aeec-72bdf8fef083"). InnerVolumeSpecName "kube-api-access-85pd8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:24:12.058320 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:12.058297 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/859f7201-3171-4fe2-aeec-72bdf8fef083-util" (OuterVolumeSpecName: "util") pod "859f7201-3171-4fe2-aeec-72bdf8fef083" (UID: "859f7201-3171-4fe2-aeec-72bdf8fef083"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:24:12.152618 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:12.152547 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85pd8\" (UniqueName: \"kubernetes.io/projected/859f7201-3171-4fe2-aeec-72bdf8fef083-kube-api-access-85pd8\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:24:12.152618 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:12.152573 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/859f7201-3171-4fe2-aeec-72bdf8fef083-bundle\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:24:12.152618 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:12.152584 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/859f7201-3171-4fe2-aeec-72bdf8fef083-util\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:24:12.857072 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:12.857032 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp" event={"ID":"859f7201-3171-4fe2-aeec-72bdf8fef083","Type":"ContainerDied","Data":"d632a02d29951a982fb29b6c8586adc50d3d9a4677644e996d8a113c21df072f"} Jul 14 11:24:12.857072 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:12.857061 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afhtnzp" Jul 14 11:24:12.857290 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:12.857098 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d632a02d29951a982fb29b6c8586adc50d3d9a4677644e996d8a113c21df072f" Jul 14 11:24:16.741600 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:16.741570 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:20.097666 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.097579 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:20.234729 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.234693 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:20.242305 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.242268 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-985446bdb-t76xq"] Jul 14 11:24:20.242727 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.242704 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="859f7201-3171-4fe2-aeec-72bdf8fef083" containerName="extract" Jul 14 11:24:20.242727 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.242727 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="859f7201-3171-4fe2-aeec-72bdf8fef083" containerName="extract" Jul 14 11:24:20.242892 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.242745 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="859f7201-3171-4fe2-aeec-72bdf8fef083" containerName="util" Jul 14 11:24:20.242892 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.242753 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="859f7201-3171-4fe2-aeec-72bdf8fef083" containerName="util" Jul 14 11:24:20.242892 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.242767 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="859f7201-3171-4fe2-aeec-72bdf8fef083" containerName="pull" Jul 14 11:24:20.242892 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.242774 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="859f7201-3171-4fe2-aeec-72bdf8fef083" containerName="pull" Jul 14 11:24:20.242892 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.242841 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="859f7201-3171-4fe2-aeec-72bdf8fef083" containerName="extract" Jul 14 11:24:20.245874 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.245859 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-985446bdb-t76xq" Jul 14 11:24:20.249826 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.249803 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 14 11:24:20.251178 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.251146 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 14 11:24:20.251329 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.251307 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 14 11:24:20.251404 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.251353 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 14 11:24:20.251404 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.251362 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 14 11:24:20.251502 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.251437 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-667wr\"" Jul 14 11:24:20.259134 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.259104 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-985446bdb-t76xq"] Jul 14 11:24:20.315825 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.315794 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f8daf19-7942-4073-8134-50e76918a5ca-cert\") pod \"lws-controller-manager-985446bdb-t76xq\" (UID: \"1f8daf19-7942-4073-8134-50e76918a5ca\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-t76xq" Jul 14 11:24:20.316057 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.316029 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5cg7\" (UniqueName: \"kubernetes.io/projected/1f8daf19-7942-4073-8134-50e76918a5ca-kube-api-access-f5cg7\") pod \"lws-controller-manager-985446bdb-t76xq\" (UID: \"1f8daf19-7942-4073-8134-50e76918a5ca\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-t76xq" Jul 14 11:24:20.316153 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.316064 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1f8daf19-7942-4073-8134-50e76918a5ca-metrics-cert\") pod \"lws-controller-manager-985446bdb-t76xq\" (UID: \"1f8daf19-7942-4073-8134-50e76918a5ca\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-t76xq" Jul 14 11:24:20.316202 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.316177 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1f8daf19-7942-4073-8134-50e76918a5ca-manager-config\") pod \"lws-controller-manager-985446bdb-t76xq\" (UID: \"1f8daf19-7942-4073-8134-50e76918a5ca\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-t76xq" Jul 14 11:24:20.416995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.416915 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1f8daf19-7942-4073-8134-50e76918a5ca-manager-config\") pod \"lws-controller-manager-985446bdb-t76xq\" (UID: \"1f8daf19-7942-4073-8134-50e76918a5ca\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-t76xq" Jul 14 11:24:20.416995 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.416959 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f8daf19-7942-4073-8134-50e76918a5ca-cert\") pod \"lws-controller-manager-985446bdb-t76xq\" (UID: \"1f8daf19-7942-4073-8134-50e76918a5ca\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-t76xq" Jul 14 11:24:20.417216 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.417077 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5cg7\" (UniqueName: \"kubernetes.io/projected/1f8daf19-7942-4073-8134-50e76918a5ca-kube-api-access-f5cg7\") pod \"lws-controller-manager-985446bdb-t76xq\" (UID: \"1f8daf19-7942-4073-8134-50e76918a5ca\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-t76xq" Jul 14 11:24:20.417216 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.417127 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1f8daf19-7942-4073-8134-50e76918a5ca-metrics-cert\") pod \"lws-controller-manager-985446bdb-t76xq\" (UID: \"1f8daf19-7942-4073-8134-50e76918a5ca\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-t76xq" Jul 14 11:24:20.417612 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.417592 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1f8daf19-7942-4073-8134-50e76918a5ca-manager-config\") pod \"lws-controller-manager-985446bdb-t76xq\" (UID: \"1f8daf19-7942-4073-8134-50e76918a5ca\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-t76xq" Jul 14 11:24:20.419457 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.419435 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1f8daf19-7942-4073-8134-50e76918a5ca-metrics-cert\") pod \"lws-controller-manager-985446bdb-t76xq\" (UID: \"1f8daf19-7942-4073-8134-50e76918a5ca\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-t76xq" Jul 14 11:24:20.419457 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.419451 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f8daf19-7942-4073-8134-50e76918a5ca-cert\") pod \"lws-controller-manager-985446bdb-t76xq\" (UID: \"1f8daf19-7942-4073-8134-50e76918a5ca\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-t76xq" Jul 14 11:24:20.426288 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.426267 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5cg7\" (UniqueName: \"kubernetes.io/projected/1f8daf19-7942-4073-8134-50e76918a5ca-kube-api-access-f5cg7\") pod \"lws-controller-manager-985446bdb-t76xq\" (UID: \"1f8daf19-7942-4073-8134-50e76918a5ca\") " pod="openshift-lws-operator/lws-controller-manager-985446bdb-t76xq" Jul 14 11:24:20.557672 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.557619 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-985446bdb-t76xq" Jul 14 11:24:20.696878 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.696855 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-985446bdb-t76xq"] Jul 14 11:24:20.699970 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:24:20.699943 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f8daf19_7942_4073_8134_50e76918a5ca.slice/crio-30603b44a477ad41ded984392538a30b76d51c9a3e4d2907ec41e23f62adb864 WatchSource:0}: Error finding container 30603b44a477ad41ded984392538a30b76d51c9a3e4d2907ec41e23f62adb864: Status 404 returned error can't find the container with id 30603b44a477ad41ded984392538a30b76d51c9a3e4d2907ec41e23f62adb864 Jul 14 11:24:20.895319 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:20.895283 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-985446bdb-t76xq" event={"ID":"1f8daf19-7942-4073-8134-50e76918a5ca","Type":"ContainerStarted","Data":"30603b44a477ad41ded984392538a30b76d51c9a3e4d2907ec41e23f62adb864"} Jul 14 11:24:22.907712 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:22.907669 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-985446bdb-t76xq" event={"ID":"1f8daf19-7942-4073-8134-50e76918a5ca","Type":"ContainerStarted","Data":"b7b5e46535f7913ece481cc2de7606ecf11375cdf21233476bea24576872bacd"} Jul 14 11:24:22.908149 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:22.907845 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-985446bdb-t76xq" Jul 14 11:24:22.930572 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:22.930521 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-985446bdb-t76xq" podStartSLOduration=1.050710416 podStartE2EDuration="2.9305044s" totalImagesPullingTime="1.879793984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:24:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:24:22.929204724 +0000 UTC m=+357.090281192" watchObservedRunningTime="2026-07-14 11:24:22.9305044 +0000 UTC m=+357.091580867" Jul 14 11:24:23.648835 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:23.648804 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:23.749797 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:23.749763 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:24.235254 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.235217 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:24.434255 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.434223 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:24.545218 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.545137 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:24.605259 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.605230 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx"] Jul 14 11:24:24.607644 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.607611 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" Jul 14 11:24:24.610844 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.610825 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 14 11:24:24.611471 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.611452 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 14 11:24:24.611659 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.611640 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gz7sx\"" Jul 14 11:24:24.611717 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.611706 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 14 11:24:24.612580 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.612566 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 14 11:24:24.629036 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.629006 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx"] Jul 14 11:24:24.757759 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.757722 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24603c08-c875-4778-a6ee-bfbabc5d8cb0-tmp\") pod \"opendatahub-operator-controller-manager-555b6fb65-qrcdx\" (UID: \"24603c08-c875-4778-a6ee-bfbabc5d8cb0\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" Jul 14 11:24:24.757909 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.757767 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwztz\" (UniqueName: \"kubernetes.io/projected/24603c08-c875-4778-a6ee-bfbabc5d8cb0-kube-api-access-lwztz\") pod \"opendatahub-operator-controller-manager-555b6fb65-qrcdx\" (UID: \"24603c08-c875-4778-a6ee-bfbabc5d8cb0\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" Jul 14 11:24:24.757909 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.757795 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/24603c08-c875-4778-a6ee-bfbabc5d8cb0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-555b6fb65-qrcdx\" (UID: \"24603c08-c875-4778-a6ee-bfbabc5d8cb0\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" Jul 14 11:24:24.757909 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.757854 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/24603c08-c875-4778-a6ee-bfbabc5d8cb0-manifests\") pod \"opendatahub-operator-controller-manager-555b6fb65-qrcdx\" (UID: \"24603c08-c875-4778-a6ee-bfbabc5d8cb0\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" Jul 14 11:24:24.757909 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.757876 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/24603c08-c875-4778-a6ee-bfbabc5d8cb0-webhook-cert\") pod \"opendatahub-operator-controller-manager-555b6fb65-qrcdx\" (UID: \"24603c08-c875-4778-a6ee-bfbabc5d8cb0\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" Jul 14 11:24:24.778830 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.778802 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:24.859025 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.859001 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/24603c08-c875-4778-a6ee-bfbabc5d8cb0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-555b6fb65-qrcdx\" (UID: \"24603c08-c875-4778-a6ee-bfbabc5d8cb0\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" Jul 14 11:24:24.859158 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.859059 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/24603c08-c875-4778-a6ee-bfbabc5d8cb0-manifests\") pod \"opendatahub-operator-controller-manager-555b6fb65-qrcdx\" (UID: \"24603c08-c875-4778-a6ee-bfbabc5d8cb0\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" Jul 14 11:24:24.859158 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.859084 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/24603c08-c875-4778-a6ee-bfbabc5d8cb0-webhook-cert\") pod \"opendatahub-operator-controller-manager-555b6fb65-qrcdx\" (UID: \"24603c08-c875-4778-a6ee-bfbabc5d8cb0\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" Jul 14 11:24:24.859158 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.859106 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24603c08-c875-4778-a6ee-bfbabc5d8cb0-tmp\") pod \"opendatahub-operator-controller-manager-555b6fb65-qrcdx\" (UID: \"24603c08-c875-4778-a6ee-bfbabc5d8cb0\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" Jul 14 11:24:24.859158 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.859126 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwztz\" (UniqueName: \"kubernetes.io/projected/24603c08-c875-4778-a6ee-bfbabc5d8cb0-kube-api-access-lwztz\") pod \"opendatahub-operator-controller-manager-555b6fb65-qrcdx\" (UID: \"24603c08-c875-4778-a6ee-bfbabc5d8cb0\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" Jul 14 11:24:24.859580 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.859554 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/24603c08-c875-4778-a6ee-bfbabc5d8cb0-manifests\") pod \"opendatahub-operator-controller-manager-555b6fb65-qrcdx\" (UID: \"24603c08-c875-4778-a6ee-bfbabc5d8cb0\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" Jul 14 11:24:24.861470 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.861445 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24603c08-c875-4778-a6ee-bfbabc5d8cb0-tmp\") pod \"opendatahub-operator-controller-manager-555b6fb65-qrcdx\" (UID: \"24603c08-c875-4778-a6ee-bfbabc5d8cb0\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" Jul 14 11:24:24.861560 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.861525 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/24603c08-c875-4778-a6ee-bfbabc5d8cb0-webhook-cert\") pod \"opendatahub-operator-controller-manager-555b6fb65-qrcdx\" (UID: \"24603c08-c875-4778-a6ee-bfbabc5d8cb0\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" Jul 14 11:24:24.861715 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.861696 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/24603c08-c875-4778-a6ee-bfbabc5d8cb0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-555b6fb65-qrcdx\" (UID: \"24603c08-c875-4778-a6ee-bfbabc5d8cb0\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" Jul 14 11:24:24.868538 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.868514 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwztz\" (UniqueName: \"kubernetes.io/projected/24603c08-c875-4778-a6ee-bfbabc5d8cb0-kube-api-access-lwztz\") pod \"opendatahub-operator-controller-manager-555b6fb65-qrcdx\" (UID: \"24603c08-c875-4778-a6ee-bfbabc5d8cb0\") " pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" Jul 14 11:24:24.881195 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.881170 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:24.920625 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.920602 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" Jul 14 11:24:24.986100 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:24.986070 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:25.064917 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:25.064890 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx"] Jul 14 11:24:25.066898 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:24:25.066874 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24603c08_c875_4778_a6ee_bfbabc5d8cb0.slice/crio-9416154ac4a138066dfe14bac586b76071a93a2afd3285a85da3f6027808947a WatchSource:0}: Error finding container 9416154ac4a138066dfe14bac586b76071a93a2afd3285a85da3f6027808947a: Status 404 returned error can't find the container with id 9416154ac4a138066dfe14bac586b76071a93a2afd3285a85da3f6027808947a Jul 14 11:24:25.083734 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:25.083712 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:25.185407 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:25.185343 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:25.928409 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:25.928370 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" event={"ID":"24603c08-c875-4778-a6ee-bfbabc5d8cb0","Type":"ContainerStarted","Data":"9416154ac4a138066dfe14bac586b76071a93a2afd3285a85da3f6027808947a"} Jul 14 11:24:28.949172 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:28.949130 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" event={"ID":"24603c08-c875-4778-a6ee-bfbabc5d8cb0","Type":"ContainerStarted","Data":"b4e225e730560d1d38a31df5b7fdb7d666552eb845100fdd20e2fe289483daa8"} Jul 14 11:24:29.955008 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:29.954972 2571 generic.go:356] "Generic (PLEG): container finished" podID="24603c08-c875-4778-a6ee-bfbabc5d8cb0" containerID="b4e225e730560d1d38a31df5b7fdb7d666552eb845100fdd20e2fe289483daa8" exitCode=0 Jul 14 11:24:29.955458 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:29.955033 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" event={"ID":"24603c08-c875-4778-a6ee-bfbabc5d8cb0","Type":"ContainerDied","Data":"b4e225e730560d1d38a31df5b7fdb7d666552eb845100fdd20e2fe289483daa8"} Jul 14 11:24:30.961287 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:30.961245 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" event={"ID":"24603c08-c875-4778-a6ee-bfbabc5d8cb0","Type":"ContainerStarted","Data":"f4bbad7c5516927d9bc035c2dfff0d459f59e41b814ef0e7b1325a2faad3d7b9"} Jul 14 11:24:30.961692 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:30.961444 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" Jul 14 11:24:30.987386 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:30.987341 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" podStartSLOduration=2.464040971 podStartE2EDuration="6.987321972s" totalImagesPullingTime="3.440733351s" totalInitContainerRuntime="1.08254765s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:24:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:24:30.985683063 +0000 UTC m=+365.146759529" watchObservedRunningTime="2026-07-14 11:24:30.987321972 +0000 UTC m=+365.148398440" Jul 14 11:24:33.914677 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:33.914618 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-985446bdb-t76xq" Jul 14 11:24:36.672253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:36.672222 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:37.098211 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:37.098181 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:37.165966 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:37.165937 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:38.328674 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:38.328602 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:38.390489 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:38.390450 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:41.968268 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:41.968240 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-555b6fb65-qrcdx" Jul 14 11:24:43.870841 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:43.870804 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:44.034819 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:44.034774 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:44.712818 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:44.712778 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:45.094039 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:45.094011 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:45.201855 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:45.201825 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:45.308484 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:45.308442 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:45.403105 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:45.403036 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:45.751031 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:45.750957 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:46.108928 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:46.108882 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:46.582127 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:46.582039 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:46.668963 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:46.668930 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:47.773710 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:47.773677 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:49.800399 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:49.800364 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:52.970112 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:52.970065 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:53.334784 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.334704 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r"] Jul 14 11:24:53.337911 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.337889 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.343131 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.343108 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 14 11:24:53.343619 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.343579 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 14 11:24:53.343774 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.343751 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-x6mvm\"" Jul 14 11:24:53.344099 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.344082 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 14 11:24:53.359962 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.359938 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r"] Jul 14 11:24:53.383783 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.383754 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:53.487614 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.487582 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e461ac53-05ac-4276-82fc-30f563afd714-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.487614 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.487617 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e461ac53-05ac-4276-82fc-30f563afd714-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.487845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.487665 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e461ac53-05ac-4276-82fc-30f563afd714-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.487845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.487692 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e461ac53-05ac-4276-82fc-30f563afd714-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.487845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.487713 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzwnj\" (UniqueName: \"kubernetes.io/projected/e461ac53-05ac-4276-82fc-30f563afd714-kube-api-access-tzwnj\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.487845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.487740 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e461ac53-05ac-4276-82fc-30f563afd714-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.487845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.487775 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e461ac53-05ac-4276-82fc-30f563afd714-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.487845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.487796 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e461ac53-05ac-4276-82fc-30f563afd714-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.488064 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.487898 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e461ac53-05ac-4276-82fc-30f563afd714-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.588540 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.588469 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e461ac53-05ac-4276-82fc-30f563afd714-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.588540 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.588503 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e461ac53-05ac-4276-82fc-30f563afd714-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.588540 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.588541 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e461ac53-05ac-4276-82fc-30f563afd714-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.588820 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.588576 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e461ac53-05ac-4276-82fc-30f563afd714-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.588820 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.588597 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e461ac53-05ac-4276-82fc-30f563afd714-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.588820 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.588613 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e461ac53-05ac-4276-82fc-30f563afd714-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.588820 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.588652 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e461ac53-05ac-4276-82fc-30f563afd714-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.588820 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.588686 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzwnj\" (UniqueName: \"kubernetes.io/projected/e461ac53-05ac-4276-82fc-30f563afd714-kube-api-access-tzwnj\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.588820 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.588716 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e461ac53-05ac-4276-82fc-30f563afd714-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.589100 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.588934 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e461ac53-05ac-4276-82fc-30f563afd714-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.589100 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.588954 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e461ac53-05ac-4276-82fc-30f563afd714-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.589100 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.589016 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e461ac53-05ac-4276-82fc-30f563afd714-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.589100 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.589073 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e461ac53-05ac-4276-82fc-30f563afd714-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.589455 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.589435 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e461ac53-05ac-4276-82fc-30f563afd714-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.590976 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.590956 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e461ac53-05ac-4276-82fc-30f563afd714-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.591415 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.591395 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e461ac53-05ac-4276-82fc-30f563afd714-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.594557 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.594525 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:53.609934 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.609914 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e461ac53-05ac-4276-82fc-30f563afd714-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.610252 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.610232 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzwnj\" (UniqueName: \"kubernetes.io/projected/e461ac53-05ac-4276-82fc-30f563afd714-kube-api-access-tzwnj\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lhp8r\" (UID: \"e461ac53-05ac-4276-82fc-30f563afd714\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.648269 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.648246 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:53.810363 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.810330 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r"] Jul 14 11:24:53.814059 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:24:53.814037 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode461ac53_05ac_4276_82fc_30f563afd714.slice/crio-9d68380649d262313430afb0b940f683ad361f129a75c70196f8694fab43e50c WatchSource:0}: Error finding container 9d68380649d262313430afb0b940f683ad361f129a75c70196f8694fab43e50c: Status 404 returned error can't find the container with id 9d68380649d262313430afb0b940f683ad361f129a75c70196f8694fab43e50c Jul 14 11:24:53.883004 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.882983 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2"] Jul 14 11:24:53.888977 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.888958 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:53.897680 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.897660 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 14 11:24:53.897784 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.897760 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 14 11:24:53.897784 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.897761 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 14 11:24:53.898492 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.898479 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-b94mv\"" Jul 14 11:24:53.913539 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.913518 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2"] Jul 14 11:24:53.992594 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.992558 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/acda5f77-8b32-4605-b298-ea11ae9e3f64-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qhzl2\" (UID: \"acda5f77-8b32-4605-b298-ea11ae9e3f64\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:53.992985 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.992606 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/acda5f77-8b32-4605-b298-ea11ae9e3f64-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qhzl2\" (UID: \"acda5f77-8b32-4605-b298-ea11ae9e3f64\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:53.992985 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.992684 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/acda5f77-8b32-4605-b298-ea11ae9e3f64-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qhzl2\" (UID: \"acda5f77-8b32-4605-b298-ea11ae9e3f64\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:53.992985 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.992726 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/acda5f77-8b32-4605-b298-ea11ae9e3f64-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qhzl2\" (UID: \"acda5f77-8b32-4605-b298-ea11ae9e3f64\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:53.992985 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.992745 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acda5f77-8b32-4605-b298-ea11ae9e3f64-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qhzl2\" (UID: \"acda5f77-8b32-4605-b298-ea11ae9e3f64\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:53.992985 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.992769 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/acda5f77-8b32-4605-b298-ea11ae9e3f64-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qhzl2\" (UID: \"acda5f77-8b32-4605-b298-ea11ae9e3f64\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:53.992985 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:53.992854 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26bjd\" (UniqueName: \"kubernetes.io/projected/acda5f77-8b32-4605-b298-ea11ae9e3f64-kube-api-access-26bjd\") pod \"istiod-openshift-gateway-94bb8fbfd-qhzl2\" (UID: \"acda5f77-8b32-4605-b298-ea11ae9e3f64\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:54.071331 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:54.071300 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" event={"ID":"e461ac53-05ac-4276-82fc-30f563afd714","Type":"ContainerStarted","Data":"9d68380649d262313430afb0b940f683ad361f129a75c70196f8694fab43e50c"} Jul 14 11:24:54.093822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:54.093791 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/acda5f77-8b32-4605-b298-ea11ae9e3f64-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qhzl2\" (UID: \"acda5f77-8b32-4605-b298-ea11ae9e3f64\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:54.093822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:54.093834 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/acda5f77-8b32-4605-b298-ea11ae9e3f64-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qhzl2\" (UID: \"acda5f77-8b32-4605-b298-ea11ae9e3f64\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:54.094030 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:54.093861 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/acda5f77-8b32-4605-b298-ea11ae9e3f64-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qhzl2\" (UID: \"acda5f77-8b32-4605-b298-ea11ae9e3f64\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:54.094030 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:54.093894 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/acda5f77-8b32-4605-b298-ea11ae9e3f64-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qhzl2\" (UID: \"acda5f77-8b32-4605-b298-ea11ae9e3f64\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:54.094030 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:54.093918 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acda5f77-8b32-4605-b298-ea11ae9e3f64-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qhzl2\" (UID: \"acda5f77-8b32-4605-b298-ea11ae9e3f64\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:54.094030 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:54.093952 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/acda5f77-8b32-4605-b298-ea11ae9e3f64-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qhzl2\" (UID: \"acda5f77-8b32-4605-b298-ea11ae9e3f64\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:54.094030 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:54.094009 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26bjd\" (UniqueName: \"kubernetes.io/projected/acda5f77-8b32-4605-b298-ea11ae9e3f64-kube-api-access-26bjd\") pod \"istiod-openshift-gateway-94bb8fbfd-qhzl2\" (UID: \"acda5f77-8b32-4605-b298-ea11ae9e3f64\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:54.094707 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:54.094679 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/acda5f77-8b32-4605-b298-ea11ae9e3f64-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qhzl2\" (UID: \"acda5f77-8b32-4605-b298-ea11ae9e3f64\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:54.096517 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:54.096488 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/acda5f77-8b32-4605-b298-ea11ae9e3f64-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qhzl2\" (UID: \"acda5f77-8b32-4605-b298-ea11ae9e3f64\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:54.096790 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:54.096768 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/acda5f77-8b32-4605-b298-ea11ae9e3f64-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qhzl2\" (UID: \"acda5f77-8b32-4605-b298-ea11ae9e3f64\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:54.096877 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:54.096836 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/acda5f77-8b32-4605-b298-ea11ae9e3f64-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qhzl2\" (UID: \"acda5f77-8b32-4605-b298-ea11ae9e3f64\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:54.096937 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:54.096916 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/acda5f77-8b32-4605-b298-ea11ae9e3f64-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qhzl2\" (UID: \"acda5f77-8b32-4605-b298-ea11ae9e3f64\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:54.106759 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:54.106700 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acda5f77-8b32-4605-b298-ea11ae9e3f64-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qhzl2\" (UID: \"acda5f77-8b32-4605-b298-ea11ae9e3f64\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:54.107022 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:54.106998 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26bjd\" (UniqueName: \"kubernetes.io/projected/acda5f77-8b32-4605-b298-ea11ae9e3f64-kube-api-access-26bjd\") pod \"istiod-openshift-gateway-94bb8fbfd-qhzl2\" (UID: \"acda5f77-8b32-4605-b298-ea11ae9e3f64\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:54.198430 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:54.198349 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:54.203580 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:54.203549 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:54.333270 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:54.333237 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2"] Jul 14 11:24:54.338056 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:24:54.338004 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacda5f77_8b32_4605_b298_ea11ae9e3f64.slice/crio-47ce226b5f33a4a607a22169d47d4178553256f87d7d90390fe314ccbd44371a WatchSource:0}: Error finding container 47ce226b5f33a4a607a22169d47d4178553256f87d7d90390fe314ccbd44371a: Status 404 returned error can't find the container with id 47ce226b5f33a4a607a22169d47d4178553256f87d7d90390fe314ccbd44371a Jul 14 11:24:55.080358 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:55.080285 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" event={"ID":"acda5f77-8b32-4605-b298-ea11ae9e3f64","Type":"ContainerStarted","Data":"47ce226b5f33a4a607a22169d47d4178553256f87d7d90390fe314ccbd44371a"} Jul 14 11:24:56.916060 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:56.916009 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:57.108217 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:57.108180 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:57.191981 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:57.191852 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:57.310958 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:57.310927 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:57.476726 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:57.476606 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:57.720714 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:57.720683 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:57.826727 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:57.826691 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:24:57.826824 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:57.826758 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:24:57.831485 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:57.831461 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:24:57.831526 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:57.831515 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:24:57.831560 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:57.831541 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:24:58.103344 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:58.103310 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" event={"ID":"e461ac53-05ac-4276-82fc-30f563afd714","Type":"ContainerStarted","Data":"05153754d2070a2c9b17deffc7bb8626a6a6dd603aa70ea7d4d70e5981b98b53"} Jul 14 11:24:58.105065 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:58.105039 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" event={"ID":"acda5f77-8b32-4605-b298-ea11ae9e3f64","Type":"ContainerStarted","Data":"cd8126b0380642d32852a23c040a6b64d8dd1d8a89a8dc1908631095cbf61da2"} Jul 14 11:24:58.105199 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:58.105103 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:58.106569 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:58.106542 2571 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-qhzl2 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 11:24:58.106700 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:58.106598 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" podUID="acda5f77-8b32-4605-b298-ea11ae9e3f64" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 11:24:58.137506 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:58.137455 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" podStartSLOduration=1.122144521 podStartE2EDuration="5.137436658s" totalImagesPullingTime="4.015292137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:24:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:24:58.137284236 +0000 UTC m=+392.298360703" watchObservedRunningTime="2026-07-14 11:24:58.137436658 +0000 UTC m=+392.298513124" Jul 14 11:24:58.171621 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:58.171551 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" podStartSLOduration=1.6852612599999999 podStartE2EDuration="5.171534119s" totalImagesPullingTime="3.486272859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:24:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:24:58.171306682 +0000 UTC m=+392.332383164" watchObservedRunningTime="2026-07-14 11:24:58.171534119 +0000 UTC m=+392.332610588" Jul 14 11:24:58.648705 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:58.648666 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:58.652729 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:58.652703 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:59.110297 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:59.110260 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:59.110748 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:59.110687 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qhzl2" Jul 14 11:24:59.111061 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:59.111048 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lhp8r" Jul 14 11:24:59.442766 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:59.442685 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:24:59.577950 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:24:59.577919 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:25:12.014164 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:25:12.014119 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:25:12.115156 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:25:12.115122 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:25:45.275311 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:25:45.275252 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:25:45.377076 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:25:45.377025 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:25:45.472604 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:25:45.472569 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:00.786407 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:00.786313 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9"] Jul 14 11:26:00.789806 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:00.789787 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9" Jul 14 11:26:00.793003 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:00.792987 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 11:26:00.793114 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:00.793035 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-82t9v\"" Jul 14 11:26:00.793165 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:00.793119 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 11:26:00.799794 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:00.799772 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9"] Jul 14 11:26:00.849683 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:00.849650 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef5afc8c-1590-43b6-b740-5768dd559481-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9\" (UID: \"ef5afc8c-1590-43b6-b740-5768dd559481\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9" Jul 14 11:26:00.849813 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:00.849712 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef5afc8c-1590-43b6-b740-5768dd559481-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9\" (UID: \"ef5afc8c-1590-43b6-b740-5768dd559481\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9" Jul 14 11:26:00.849813 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:00.849754 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xplft\" (UniqueName: \"kubernetes.io/projected/ef5afc8c-1590-43b6-b740-5768dd559481-kube-api-access-xplft\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9\" (UID: \"ef5afc8c-1590-43b6-b740-5768dd559481\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9" Jul 14 11:26:00.950665 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:00.950615 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef5afc8c-1590-43b6-b740-5768dd559481-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9\" (UID: \"ef5afc8c-1590-43b6-b740-5768dd559481\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9" Jul 14 11:26:00.950799 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:00.950706 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef5afc8c-1590-43b6-b740-5768dd559481-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9\" (UID: \"ef5afc8c-1590-43b6-b740-5768dd559481\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9" Jul 14 11:26:00.950799 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:00.950726 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xplft\" (UniqueName: \"kubernetes.io/projected/ef5afc8c-1590-43b6-b740-5768dd559481-kube-api-access-xplft\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9\" (UID: \"ef5afc8c-1590-43b6-b740-5768dd559481\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9" Jul 14 11:26:00.951049 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:00.951027 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef5afc8c-1590-43b6-b740-5768dd559481-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9\" (UID: \"ef5afc8c-1590-43b6-b740-5768dd559481\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9" Jul 14 11:26:00.951115 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:00.951039 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef5afc8c-1590-43b6-b740-5768dd559481-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9\" (UID: \"ef5afc8c-1590-43b6-b740-5768dd559481\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9" Jul 14 11:26:00.961805 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:00.961777 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xplft\" (UniqueName: \"kubernetes.io/projected/ef5afc8c-1590-43b6-b740-5768dd559481-kube-api-access-xplft\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9\" (UID: \"ef5afc8c-1590-43b6-b740-5768dd559481\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9" Jul 14 11:26:01.099486 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.099462 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9" Jul 14 11:26:01.226430 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.226402 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9"] Jul 14 11:26:01.228312 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:26:01.228288 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef5afc8c_1590_43b6_b740_5768dd559481.slice/crio-a12e7c52598b3a49bdd8f18e889757b80cf811ddaeb86fabfd5c60d86df7fc96 WatchSource:0}: Error finding container a12e7c52598b3a49bdd8f18e889757b80cf811ddaeb86fabfd5c60d86df7fc96: Status 404 returned error can't find the container with id a12e7c52598b3a49bdd8f18e889757b80cf811ddaeb86fabfd5c60d86df7fc96 Jul 14 11:26:01.306368 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.306343 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79dbc94c75-fvgsh"] Jul 14 11:26:01.309598 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.309572 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.324131 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.324103 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79dbc94c75-fvgsh"] Jul 14 11:26:01.354988 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.354937 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e8ba38c-6de8-4040-8a62-15d7dda90c88-trusted-ca-bundle\") pod \"console-79dbc94c75-fvgsh\" (UID: \"3e8ba38c-6de8-4040-8a62-15d7dda90c88\") " pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.354988 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.354968 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e8ba38c-6de8-4040-8a62-15d7dda90c88-console-oauth-config\") pod \"console-79dbc94c75-fvgsh\" (UID: \"3e8ba38c-6de8-4040-8a62-15d7dda90c88\") " pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.355120 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.355002 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e8ba38c-6de8-4040-8a62-15d7dda90c88-console-config\") pod \"console-79dbc94c75-fvgsh\" (UID: \"3e8ba38c-6de8-4040-8a62-15d7dda90c88\") " pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.355120 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.355073 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e8ba38c-6de8-4040-8a62-15d7dda90c88-oauth-serving-cert\") pod \"console-79dbc94c75-fvgsh\" (UID: \"3e8ba38c-6de8-4040-8a62-15d7dda90c88\") " pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.355120 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.355115 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e8ba38c-6de8-4040-8a62-15d7dda90c88-service-ca\") pod \"console-79dbc94c75-fvgsh\" (UID: \"3e8ba38c-6de8-4040-8a62-15d7dda90c88\") " pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.355212 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.355146 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e8ba38c-6de8-4040-8a62-15d7dda90c88-console-serving-cert\") pod \"console-79dbc94c75-fvgsh\" (UID: \"3e8ba38c-6de8-4040-8a62-15d7dda90c88\") " pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.355212 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.355179 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsrhh\" (UniqueName: \"kubernetes.io/projected/3e8ba38c-6de8-4040-8a62-15d7dda90c88-kube-api-access-jsrhh\") pod \"console-79dbc94c75-fvgsh\" (UID: \"3e8ba38c-6de8-4040-8a62-15d7dda90c88\") " pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.361833 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.361812 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:01.388077 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.388047 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j"] Jul 14 11:26:01.394432 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.393257 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j" Jul 14 11:26:01.401587 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.401561 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j"] Jul 14 11:26:01.420263 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.403857 2571 generic.go:356] "Generic (PLEG): container finished" podID="ef5afc8c-1590-43b6-b740-5768dd559481" containerID="342bc3415f8b449684c314d79791522f85b141f12d2e71fa1dfefcaf12e7a127" exitCode=0 Jul 14 11:26:01.420263 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.403999 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9" event={"ID":"ef5afc8c-1590-43b6-b740-5768dd559481","Type":"ContainerDied","Data":"342bc3415f8b449684c314d79791522f85b141f12d2e71fa1dfefcaf12e7a127"} Jul 14 11:26:01.420263 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.404035 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9" event={"ID":"ef5afc8c-1590-43b6-b740-5768dd559481","Type":"ContainerStarted","Data":"a12e7c52598b3a49bdd8f18e889757b80cf811ddaeb86fabfd5c60d86df7fc96"} Jul 14 11:26:01.455591 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.455561 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e8ba38c-6de8-4040-8a62-15d7dda90c88-console-config\") pod \"console-79dbc94c75-fvgsh\" (UID: \"3e8ba38c-6de8-4040-8a62-15d7dda90c88\") " pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.455756 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.455614 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc697cb2-85ad-4af1-b87f-7f30203b1bd2-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j\" (UID: \"cc697cb2-85ad-4af1-b87f-7f30203b1bd2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j" Jul 14 11:26:01.455756 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.455650 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e8ba38c-6de8-4040-8a62-15d7dda90c88-oauth-serving-cert\") pod \"console-79dbc94c75-fvgsh\" (UID: \"3e8ba38c-6de8-4040-8a62-15d7dda90c88\") " pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.455756 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.455693 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e8ba38c-6de8-4040-8a62-15d7dda90c88-service-ca\") pod \"console-79dbc94c75-fvgsh\" (UID: \"3e8ba38c-6de8-4040-8a62-15d7dda90c88\") " pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.455756 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.455727 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e8ba38c-6de8-4040-8a62-15d7dda90c88-console-serving-cert\") pod \"console-79dbc94c75-fvgsh\" (UID: \"3e8ba38c-6de8-4040-8a62-15d7dda90c88\") " pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.456031 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.455781 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsrhh\" (UniqueName: \"kubernetes.io/projected/3e8ba38c-6de8-4040-8a62-15d7dda90c88-kube-api-access-jsrhh\") pod \"console-79dbc94c75-fvgsh\" (UID: \"3e8ba38c-6de8-4040-8a62-15d7dda90c88\") " pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.456031 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.455816 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kslpf\" (UniqueName: \"kubernetes.io/projected/cc697cb2-85ad-4af1-b87f-7f30203b1bd2-kube-api-access-kslpf\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j\" (UID: \"cc697cb2-85ad-4af1-b87f-7f30203b1bd2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j" Jul 14 11:26:01.456031 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.455859 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e8ba38c-6de8-4040-8a62-15d7dda90c88-trusted-ca-bundle\") pod \"console-79dbc94c75-fvgsh\" (UID: \"3e8ba38c-6de8-4040-8a62-15d7dda90c88\") " pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.456031 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.455887 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e8ba38c-6de8-4040-8a62-15d7dda90c88-console-oauth-config\") pod \"console-79dbc94c75-fvgsh\" (UID: \"3e8ba38c-6de8-4040-8a62-15d7dda90c88\") " pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.456031 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.455930 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc697cb2-85ad-4af1-b87f-7f30203b1bd2-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j\" (UID: \"cc697cb2-85ad-4af1-b87f-7f30203b1bd2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j" Jul 14 11:26:01.456431 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.456404 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e8ba38c-6de8-4040-8a62-15d7dda90c88-console-config\") pod \"console-79dbc94c75-fvgsh\" (UID: \"3e8ba38c-6de8-4040-8a62-15d7dda90c88\") " pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.456515 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.456458 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e8ba38c-6de8-4040-8a62-15d7dda90c88-oauth-serving-cert\") pod \"console-79dbc94c75-fvgsh\" (UID: \"3e8ba38c-6de8-4040-8a62-15d7dda90c88\") " pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.456515 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.456500 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e8ba38c-6de8-4040-8a62-15d7dda90c88-service-ca\") pod \"console-79dbc94c75-fvgsh\" (UID: \"3e8ba38c-6de8-4040-8a62-15d7dda90c88\") " pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.457051 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.457030 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e8ba38c-6de8-4040-8a62-15d7dda90c88-trusted-ca-bundle\") pod \"console-79dbc94c75-fvgsh\" (UID: \"3e8ba38c-6de8-4040-8a62-15d7dda90c88\") " pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.458363 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.458344 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e8ba38c-6de8-4040-8a62-15d7dda90c88-console-oauth-config\") pod \"console-79dbc94c75-fvgsh\" (UID: \"3e8ba38c-6de8-4040-8a62-15d7dda90c88\") " pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.458507 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.458480 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e8ba38c-6de8-4040-8a62-15d7dda90c88-console-serving-cert\") pod \"console-79dbc94c75-fvgsh\" (UID: \"3e8ba38c-6de8-4040-8a62-15d7dda90c88\") " pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.465193 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.465177 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsrhh\" (UniqueName: \"kubernetes.io/projected/3e8ba38c-6de8-4040-8a62-15d7dda90c88-kube-api-access-jsrhh\") pod \"console-79dbc94c75-fvgsh\" (UID: \"3e8ba38c-6de8-4040-8a62-15d7dda90c88\") " pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.557031 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.557007 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc697cb2-85ad-4af1-b87f-7f30203b1bd2-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j\" (UID: \"cc697cb2-85ad-4af1-b87f-7f30203b1bd2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j" Jul 14 11:26:01.557187 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.557101 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kslpf\" (UniqueName: \"kubernetes.io/projected/cc697cb2-85ad-4af1-b87f-7f30203b1bd2-kube-api-access-kslpf\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j\" (UID: \"cc697cb2-85ad-4af1-b87f-7f30203b1bd2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j" Jul 14 11:26:01.557252 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.557239 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc697cb2-85ad-4af1-b87f-7f30203b1bd2-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j\" (UID: \"cc697cb2-85ad-4af1-b87f-7f30203b1bd2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j" Jul 14 11:26:01.557379 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.557348 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc697cb2-85ad-4af1-b87f-7f30203b1bd2-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j\" (UID: \"cc697cb2-85ad-4af1-b87f-7f30203b1bd2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j" Jul 14 11:26:01.557549 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.557528 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc697cb2-85ad-4af1-b87f-7f30203b1bd2-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j\" (UID: \"cc697cb2-85ad-4af1-b87f-7f30203b1bd2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j" Jul 14 11:26:01.566852 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.566830 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kslpf\" (UniqueName: \"kubernetes.io/projected/cc697cb2-85ad-4af1-b87f-7f30203b1bd2-kube-api-access-kslpf\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j\" (UID: \"cc697cb2-85ad-4af1-b87f-7f30203b1bd2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j" Jul 14 11:26:01.618735 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.618678 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:01.725951 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.725922 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j" Jul 14 11:26:01.744290 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.744266 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79dbc94c75-fvgsh"] Jul 14 11:26:01.745852 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:26:01.745821 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e8ba38c_6de8_4040_8a62_15d7dda90c88.slice/crio-9265c5e6b82d692401d7ffa9af32bbba350a7a575ac7ef7656906c83716a54f1 WatchSource:0}: Error finding container 9265c5e6b82d692401d7ffa9af32bbba350a7a575ac7ef7656906c83716a54f1: Status 404 returned error can't find the container with id 9265c5e6b82d692401d7ffa9af32bbba350a7a575ac7ef7656906c83716a54f1 Jul 14 11:26:01.788342 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.788318 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j"] Jul 14 11:26:01.792966 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.792949 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j" Jul 14 11:26:01.800417 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.800396 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j"] Jul 14 11:26:01.860370 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.860325 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5llj\" (UniqueName: \"kubernetes.io/projected/5685eb77-c028-43e8-8532-20e60bf76588-kube-api-access-m5llj\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j\" (UID: \"5685eb77-c028-43e8-8532-20e60bf76588\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j" Jul 14 11:26:01.860548 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.860469 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5685eb77-c028-43e8-8532-20e60bf76588-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j\" (UID: \"5685eb77-c028-43e8-8532-20e60bf76588\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j" Jul 14 11:26:01.860548 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.860506 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5685eb77-c028-43e8-8532-20e60bf76588-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j\" (UID: \"5685eb77-c028-43e8-8532-20e60bf76588\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j" Jul 14 11:26:01.870833 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.870809 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j"] Jul 14 11:26:01.873053 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:26:01.873028 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc697cb2_85ad_4af1_b87f_7f30203b1bd2.slice/crio-9a229d8271a8db432106eb8878222197e476145dfe4b45c966c61e06e4725f79 WatchSource:0}: Error finding container 9a229d8271a8db432106eb8878222197e476145dfe4b45c966c61e06e4725f79: Status 404 returned error can't find the container with id 9a229d8271a8db432106eb8878222197e476145dfe4b45c966c61e06e4725f79 Jul 14 11:26:01.961692 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.961658 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5685eb77-c028-43e8-8532-20e60bf76588-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j\" (UID: \"5685eb77-c028-43e8-8532-20e60bf76588\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j" Jul 14 11:26:01.961840 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.961708 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5685eb77-c028-43e8-8532-20e60bf76588-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j\" (UID: \"5685eb77-c028-43e8-8532-20e60bf76588\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j" Jul 14 11:26:01.961840 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.961761 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5llj\" (UniqueName: \"kubernetes.io/projected/5685eb77-c028-43e8-8532-20e60bf76588-kube-api-access-m5llj\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j\" (UID: \"5685eb77-c028-43e8-8532-20e60bf76588\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j" Jul 14 11:26:01.962189 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.962161 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5685eb77-c028-43e8-8532-20e60bf76588-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j\" (UID: \"5685eb77-c028-43e8-8532-20e60bf76588\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j" Jul 14 11:26:01.962285 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.962167 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5685eb77-c028-43e8-8532-20e60bf76588-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j\" (UID: \"5685eb77-c028-43e8-8532-20e60bf76588\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j" Jul 14 11:26:01.971211 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:01.971186 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5llj\" (UniqueName: \"kubernetes.io/projected/5685eb77-c028-43e8-8532-20e60bf76588-kube-api-access-m5llj\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j\" (UID: \"5685eb77-c028-43e8-8532-20e60bf76588\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j" Jul 14 11:26:02.104115 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.104082 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j" Jul 14 11:26:02.186241 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.186213 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp"] Jul 14 11:26:02.191393 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.191374 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp" Jul 14 11:26:02.197896 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.197847 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp"] Jul 14 11:26:02.254381 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.254339 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j"] Jul 14 11:26:02.265793 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.265309 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fbe24769-cda3-4f3a-a912-b13763402dd9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp\" (UID: \"fbe24769-cda3-4f3a-a912-b13763402dd9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp" Jul 14 11:26:02.265793 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.265408 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlb6c\" (UniqueName: \"kubernetes.io/projected/fbe24769-cda3-4f3a-a912-b13763402dd9-kube-api-access-zlb6c\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp\" (UID: \"fbe24769-cda3-4f3a-a912-b13763402dd9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp" Jul 14 11:26:02.265793 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.265461 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fbe24769-cda3-4f3a-a912-b13763402dd9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp\" (UID: \"fbe24769-cda3-4f3a-a912-b13763402dd9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp" Jul 14 11:26:02.366154 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.366129 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlb6c\" (UniqueName: \"kubernetes.io/projected/fbe24769-cda3-4f3a-a912-b13763402dd9-kube-api-access-zlb6c\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp\" (UID: \"fbe24769-cda3-4f3a-a912-b13763402dd9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp" Jul 14 11:26:02.366299 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.366178 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fbe24769-cda3-4f3a-a912-b13763402dd9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp\" (UID: \"fbe24769-cda3-4f3a-a912-b13763402dd9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp" Jul 14 11:26:02.366299 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.366267 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fbe24769-cda3-4f3a-a912-b13763402dd9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp\" (UID: \"fbe24769-cda3-4f3a-a912-b13763402dd9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp" Jul 14 11:26:02.366735 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.366712 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fbe24769-cda3-4f3a-a912-b13763402dd9-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp\" (UID: \"fbe24769-cda3-4f3a-a912-b13763402dd9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp" Jul 14 11:26:02.366787 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.366720 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fbe24769-cda3-4f3a-a912-b13763402dd9-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp\" (UID: \"fbe24769-cda3-4f3a-a912-b13763402dd9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp" Jul 14 11:26:02.375994 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.375942 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlb6c\" (UniqueName: \"kubernetes.io/projected/fbe24769-cda3-4f3a-a912-b13763402dd9-kube-api-access-zlb6c\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp\" (UID: \"fbe24769-cda3-4f3a-a912-b13763402dd9\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp" Jul 14 11:26:02.409609 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.409585 2571 generic.go:356] "Generic (PLEG): container finished" podID="cc697cb2-85ad-4af1-b87f-7f30203b1bd2" containerID="634db74f1ebc23892ce80caf32e4eff045d4256102258afbc8cbea3c4b81c13a" exitCode=0 Jul 14 11:26:02.409715 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.409663 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j" event={"ID":"cc697cb2-85ad-4af1-b87f-7f30203b1bd2","Type":"ContainerDied","Data":"634db74f1ebc23892ce80caf32e4eff045d4256102258afbc8cbea3c4b81c13a"} Jul 14 11:26:02.409715 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.409706 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j" event={"ID":"cc697cb2-85ad-4af1-b87f-7f30203b1bd2","Type":"ContainerStarted","Data":"9a229d8271a8db432106eb8878222197e476145dfe4b45c966c61e06e4725f79"} Jul 14 11:26:02.411258 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.411235 2571 generic.go:356] "Generic (PLEG): container finished" podID="5685eb77-c028-43e8-8532-20e60bf76588" containerID="55fd0a1688f1c6653d70bb8f8a9c972ff594d122b8ccfb25842d270c9bfc5a38" exitCode=0 Jul 14 11:26:02.411331 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.411312 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j" event={"ID":"5685eb77-c028-43e8-8532-20e60bf76588","Type":"ContainerDied","Data":"55fd0a1688f1c6653d70bb8f8a9c972ff594d122b8ccfb25842d270c9bfc5a38"} Jul 14 11:26:02.411375 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.411353 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j" event={"ID":"5685eb77-c028-43e8-8532-20e60bf76588","Type":"ContainerStarted","Data":"77dc2544bc1d8364acb8afd14ae02a48a3b6140fa91753bb4bfa8349e87c87c7"} Jul 14 11:26:02.413224 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.413198 2571 generic.go:356] "Generic (PLEG): container finished" podID="ef5afc8c-1590-43b6-b740-5768dd559481" containerID="52730e5cc552fbca24b44104b3c465283361a040133a70b1c571c1d784cef284" exitCode=0 Jul 14 11:26:02.413326 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.413226 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9" event={"ID":"ef5afc8c-1590-43b6-b740-5768dd559481","Type":"ContainerDied","Data":"52730e5cc552fbca24b44104b3c465283361a040133a70b1c571c1d784cef284"} Jul 14 11:26:02.415018 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.414996 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79dbc94c75-fvgsh" event={"ID":"3e8ba38c-6de8-4040-8a62-15d7dda90c88","Type":"ContainerStarted","Data":"0286c4dbd085d9309aedead569fbf03ef9b895bee538976a2461dd1101a5aba9"} Jul 14 11:26:02.415102 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.415035 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79dbc94c75-fvgsh" event={"ID":"3e8ba38c-6de8-4040-8a62-15d7dda90c88","Type":"ContainerStarted","Data":"9265c5e6b82d692401d7ffa9af32bbba350a7a575ac7ef7656906c83716a54f1"} Jul 14 11:26:02.489211 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.489151 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79dbc94c75-fvgsh" podStartSLOduration=1.489131795 podStartE2EDuration="1.489131795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:26:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:26:02.488399542 +0000 UTC m=+456.649476009" watchObservedRunningTime="2026-07-14 11:26:02.489131795 +0000 UTC m=+456.650208264" Jul 14 11:26:02.517648 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.517612 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp" Jul 14 11:26:02.646184 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:02.646160 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp"] Jul 14 11:26:02.647334 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:26:02.647309 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbe24769_cda3_4f3a_a912_b13763402dd9.slice/crio-0ea24a0708cca8e0880b34895556408aa98ee2f55c7e7e3342ff77f52a66e7f9 WatchSource:0}: Error finding container 0ea24a0708cca8e0880b34895556408aa98ee2f55c7e7e3342ff77f52a66e7f9: Status 404 returned error can't find the container with id 0ea24a0708cca8e0880b34895556408aa98ee2f55c7e7e3342ff77f52a66e7f9 Jul 14 11:26:03.422414 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:03.422386 2571 generic.go:356] "Generic (PLEG): container finished" podID="ef5afc8c-1590-43b6-b740-5768dd559481" containerID="a8ae3c0b0a82316f4e3ef49b843cccbd5f7a2d0197fbfed829f8b5c3187dfa25" exitCode=0 Jul 14 11:26:03.422768 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:03.422475 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9" event={"ID":"ef5afc8c-1590-43b6-b740-5768dd559481","Type":"ContainerDied","Data":"a8ae3c0b0a82316f4e3ef49b843cccbd5f7a2d0197fbfed829f8b5c3187dfa25"} Jul 14 11:26:03.423837 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:03.423818 2571 generic.go:356] "Generic (PLEG): container finished" podID="fbe24769-cda3-4f3a-a912-b13763402dd9" containerID="4910516255a2ee8576f6149ccf5442027a2c6f817be9dd6ae31eca1768f88a17" exitCode=0 Jul 14 11:26:03.423954 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:03.423906 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp" event={"ID":"fbe24769-cda3-4f3a-a912-b13763402dd9","Type":"ContainerDied","Data":"4910516255a2ee8576f6149ccf5442027a2c6f817be9dd6ae31eca1768f88a17"} Jul 14 11:26:03.423954 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:03.423945 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp" event={"ID":"fbe24769-cda3-4f3a-a912-b13763402dd9","Type":"ContainerStarted","Data":"0ea24a0708cca8e0880b34895556408aa98ee2f55c7e7e3342ff77f52a66e7f9"} Jul 14 11:26:03.425793 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:03.425775 2571 generic.go:356] "Generic (PLEG): container finished" podID="cc697cb2-85ad-4af1-b87f-7f30203b1bd2" containerID="d080ad4b6f3ccfabc7392b366c131a3a626ff1752146078372fd98197b687bb6" exitCode=0 Jul 14 11:26:03.425876 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:03.425822 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j" event={"ID":"cc697cb2-85ad-4af1-b87f-7f30203b1bd2","Type":"ContainerDied","Data":"d080ad4b6f3ccfabc7392b366c131a3a626ff1752146078372fd98197b687bb6"} Jul 14 11:26:03.427775 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:03.427757 2571 generic.go:356] "Generic (PLEG): container finished" podID="5685eb77-c028-43e8-8532-20e60bf76588" containerID="bea559c2b7d49f1b7d45438b998cc9fac347f39ca38a8b15e6a81a85336dc50d" exitCode=0 Jul 14 11:26:03.427851 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:03.427803 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j" event={"ID":"5685eb77-c028-43e8-8532-20e60bf76588","Type":"ContainerDied","Data":"bea559c2b7d49f1b7d45438b998cc9fac347f39ca38a8b15e6a81a85336dc50d"} Jul 14 11:26:04.435278 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:04.435253 2571 generic.go:356] "Generic (PLEG): container finished" podID="cc697cb2-85ad-4af1-b87f-7f30203b1bd2" containerID="ee3ec02b4bbf706d6d5a639bb097eee28a869d821c5ec82af1331e146aca9021" exitCode=0 Jul 14 11:26:04.435586 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:04.435312 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j" event={"ID":"cc697cb2-85ad-4af1-b87f-7f30203b1bd2","Type":"ContainerDied","Data":"ee3ec02b4bbf706d6d5a639bb097eee28a869d821c5ec82af1331e146aca9021"} Jul 14 11:26:04.437388 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:04.437364 2571 generic.go:356] "Generic (PLEG): container finished" podID="5685eb77-c028-43e8-8532-20e60bf76588" containerID="368c6818f5ec07271be67972301ef9ea15f11f9c1a630784dd74833df2b2d0b5" exitCode=0 Jul 14 11:26:04.437472 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:04.437448 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j" event={"ID":"5685eb77-c028-43e8-8532-20e60bf76588","Type":"ContainerDied","Data":"368c6818f5ec07271be67972301ef9ea15f11f9c1a630784dd74833df2b2d0b5"} Jul 14 11:26:04.439162 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:04.439142 2571 generic.go:356] "Generic (PLEG): container finished" podID="fbe24769-cda3-4f3a-a912-b13763402dd9" containerID="ac977e9d276253d50757d9adb4b253ea5c8c56e2c484dbd0b4614de65039aff5" exitCode=0 Jul 14 11:26:04.439239 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:04.439208 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp" event={"ID":"fbe24769-cda3-4f3a-a912-b13763402dd9","Type":"ContainerDied","Data":"ac977e9d276253d50757d9adb4b253ea5c8c56e2c484dbd0b4614de65039aff5"} Jul 14 11:26:04.572496 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:04.572473 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9" Jul 14 11:26:04.690078 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:04.690052 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef5afc8c-1590-43b6-b740-5768dd559481-bundle\") pod \"ef5afc8c-1590-43b6-b740-5768dd559481\" (UID: \"ef5afc8c-1590-43b6-b740-5768dd559481\") " Jul 14 11:26:04.690217 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:04.690118 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef5afc8c-1590-43b6-b740-5768dd559481-util\") pod \"ef5afc8c-1590-43b6-b740-5768dd559481\" (UID: \"ef5afc8c-1590-43b6-b740-5768dd559481\") " Jul 14 11:26:04.690217 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:04.690152 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xplft\" (UniqueName: \"kubernetes.io/projected/ef5afc8c-1590-43b6-b740-5768dd559481-kube-api-access-xplft\") pod \"ef5afc8c-1590-43b6-b740-5768dd559481\" (UID: \"ef5afc8c-1590-43b6-b740-5768dd559481\") " Jul 14 11:26:04.690548 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:04.690516 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef5afc8c-1590-43b6-b740-5768dd559481-bundle" (OuterVolumeSpecName: "bundle") pod "ef5afc8c-1590-43b6-b740-5768dd559481" (UID: "ef5afc8c-1590-43b6-b740-5768dd559481"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:26:04.692201 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:04.692179 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef5afc8c-1590-43b6-b740-5768dd559481-kube-api-access-xplft" (OuterVolumeSpecName: "kube-api-access-xplft") pod "ef5afc8c-1590-43b6-b740-5768dd559481" (UID: "ef5afc8c-1590-43b6-b740-5768dd559481"). InnerVolumeSpecName "kube-api-access-xplft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:26:04.695105 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:04.695085 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef5afc8c-1590-43b6-b740-5768dd559481-util" (OuterVolumeSpecName: "util") pod "ef5afc8c-1590-43b6-b740-5768dd559481" (UID: "ef5afc8c-1590-43b6-b740-5768dd559481"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:26:04.791456 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:04.791397 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ef5afc8c-1590-43b6-b740-5768dd559481-util\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:26:04.791456 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:04.791419 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xplft\" (UniqueName: \"kubernetes.io/projected/ef5afc8c-1590-43b6-b740-5768dd559481-kube-api-access-xplft\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:26:04.791456 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:04.791429 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ef5afc8c-1590-43b6-b740-5768dd559481-bundle\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:26:05.445682 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.445657 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9" Jul 14 11:26:05.446126 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.445654 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9" event={"ID":"ef5afc8c-1590-43b6-b740-5768dd559481","Type":"ContainerDied","Data":"a12e7c52598b3a49bdd8f18e889757b80cf811ddaeb86fabfd5c60d86df7fc96"} Jul 14 11:26:05.446126 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.445781 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a12e7c52598b3a49bdd8f18e889757b80cf811ddaeb86fabfd5c60d86df7fc96" Jul 14 11:26:05.447781 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.447757 2571 generic.go:356] "Generic (PLEG): container finished" podID="fbe24769-cda3-4f3a-a912-b13763402dd9" containerID="0e650f07a0212a60fafef7322b41fbaf0549b3e3d744ef25c825a5d1ac7630a0" exitCode=0 Jul 14 11:26:05.447867 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.447841 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp" event={"ID":"fbe24769-cda3-4f3a-a912-b13763402dd9","Type":"ContainerDied","Data":"0e650f07a0212a60fafef7322b41fbaf0549b3e3d744ef25c825a5d1ac7630a0"} Jul 14 11:26:05.579547 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.579524 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j" Jul 14 11:26:05.610920 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.610898 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j" Jul 14 11:26:05.698968 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.698889 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5685eb77-c028-43e8-8532-20e60bf76588-util\") pod \"5685eb77-c028-43e8-8532-20e60bf76588\" (UID: \"5685eb77-c028-43e8-8532-20e60bf76588\") " Jul 14 11:26:05.698968 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.698936 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5685eb77-c028-43e8-8532-20e60bf76588-bundle\") pod \"5685eb77-c028-43e8-8532-20e60bf76588\" (UID: \"5685eb77-c028-43e8-8532-20e60bf76588\") " Jul 14 11:26:05.698968 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.698962 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5llj\" (UniqueName: \"kubernetes.io/projected/5685eb77-c028-43e8-8532-20e60bf76588-kube-api-access-m5llj\") pod \"5685eb77-c028-43e8-8532-20e60bf76588\" (UID: \"5685eb77-c028-43e8-8532-20e60bf76588\") " Jul 14 11:26:05.699220 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.698995 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc697cb2-85ad-4af1-b87f-7f30203b1bd2-bundle\") pod \"cc697cb2-85ad-4af1-b87f-7f30203b1bd2\" (UID: \"cc697cb2-85ad-4af1-b87f-7f30203b1bd2\") " Jul 14 11:26:05.699220 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.699029 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kslpf\" (UniqueName: \"kubernetes.io/projected/cc697cb2-85ad-4af1-b87f-7f30203b1bd2-kube-api-access-kslpf\") pod \"cc697cb2-85ad-4af1-b87f-7f30203b1bd2\" (UID: \"cc697cb2-85ad-4af1-b87f-7f30203b1bd2\") " Jul 14 11:26:05.699220 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.699073 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc697cb2-85ad-4af1-b87f-7f30203b1bd2-util\") pod \"cc697cb2-85ad-4af1-b87f-7f30203b1bd2\" (UID: \"cc697cb2-85ad-4af1-b87f-7f30203b1bd2\") " Jul 14 11:26:05.699568 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.699531 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc697cb2-85ad-4af1-b87f-7f30203b1bd2-bundle" (OuterVolumeSpecName: "bundle") pod "cc697cb2-85ad-4af1-b87f-7f30203b1bd2" (UID: "cc697cb2-85ad-4af1-b87f-7f30203b1bd2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:26:05.699712 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.699590 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5685eb77-c028-43e8-8532-20e60bf76588-bundle" (OuterVolumeSpecName: "bundle") pod "5685eb77-c028-43e8-8532-20e60bf76588" (UID: "5685eb77-c028-43e8-8532-20e60bf76588"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:26:05.701554 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.701522 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc697cb2-85ad-4af1-b87f-7f30203b1bd2-kube-api-access-kslpf" (OuterVolumeSpecName: "kube-api-access-kslpf") pod "cc697cb2-85ad-4af1-b87f-7f30203b1bd2" (UID: "cc697cb2-85ad-4af1-b87f-7f30203b1bd2"). InnerVolumeSpecName "kube-api-access-kslpf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:26:05.701554 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.701537 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5685eb77-c028-43e8-8532-20e60bf76588-kube-api-access-m5llj" (OuterVolumeSpecName: "kube-api-access-m5llj") pod "5685eb77-c028-43e8-8532-20e60bf76588" (UID: "5685eb77-c028-43e8-8532-20e60bf76588"). InnerVolumeSpecName "kube-api-access-m5llj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:26:05.705371 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.705347 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc697cb2-85ad-4af1-b87f-7f30203b1bd2-util" (OuterVolumeSpecName: "util") pod "cc697cb2-85ad-4af1-b87f-7f30203b1bd2" (UID: "cc697cb2-85ad-4af1-b87f-7f30203b1bd2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:26:05.705816 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.705794 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5685eb77-c028-43e8-8532-20e60bf76588-util" (OuterVolumeSpecName: "util") pod "5685eb77-c028-43e8-8532-20e60bf76588" (UID: "5685eb77-c028-43e8-8532-20e60bf76588"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:26:05.800620 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.800583 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5685eb77-c028-43e8-8532-20e60bf76588-util\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:26:05.800620 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.800617 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5685eb77-c028-43e8-8532-20e60bf76588-bundle\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:26:05.800795 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.800652 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5llj\" (UniqueName: \"kubernetes.io/projected/5685eb77-c028-43e8-8532-20e60bf76588-kube-api-access-m5llj\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:26:05.800795 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.800667 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc697cb2-85ad-4af1-b87f-7f30203b1bd2-bundle\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:26:05.800795 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.800679 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kslpf\" (UniqueName: \"kubernetes.io/projected/cc697cb2-85ad-4af1-b87f-7f30203b1bd2-kube-api-access-kslpf\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:26:05.800795 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:05.800689 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc697cb2-85ad-4af1-b87f-7f30203b1bd2-util\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:26:06.454315 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:06.454278 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j" Jul 14 11:26:06.454708 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:06.454277 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j" event={"ID":"cc697cb2-85ad-4af1-b87f-7f30203b1bd2","Type":"ContainerDied","Data":"9a229d8271a8db432106eb8878222197e476145dfe4b45c966c61e06e4725f79"} Jul 14 11:26:06.454708 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:06.454406 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9a229d8271a8db432106eb8878222197e476145dfe4b45c966c61e06e4725f79" Jul 14 11:26:06.456109 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:06.456078 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j" Jul 14 11:26:06.456225 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:06.456130 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j" event={"ID":"5685eb77-c028-43e8-8532-20e60bf76588","Type":"ContainerDied","Data":"77dc2544bc1d8364acb8afd14ae02a48a3b6140fa91753bb4bfa8349e87c87c7"} Jul 14 11:26:06.456225 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:06.456173 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="77dc2544bc1d8364acb8afd14ae02a48a3b6140fa91753bb4bfa8349e87c87c7" Jul 14 11:26:06.583599 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:06.583568 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp" Jul 14 11:26:06.708810 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:06.708723 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fbe24769-cda3-4f3a-a912-b13763402dd9-bundle\") pod \"fbe24769-cda3-4f3a-a912-b13763402dd9\" (UID: \"fbe24769-cda3-4f3a-a912-b13763402dd9\") " Jul 14 11:26:06.708810 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:06.708790 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlb6c\" (UniqueName: \"kubernetes.io/projected/fbe24769-cda3-4f3a-a912-b13763402dd9-kube-api-access-zlb6c\") pod \"fbe24769-cda3-4f3a-a912-b13763402dd9\" (UID: \"fbe24769-cda3-4f3a-a912-b13763402dd9\") " Jul 14 11:26:06.709030 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:06.708815 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fbe24769-cda3-4f3a-a912-b13763402dd9-util\") pod \"fbe24769-cda3-4f3a-a912-b13763402dd9\" (UID: \"fbe24769-cda3-4f3a-a912-b13763402dd9\") " Jul 14 11:26:06.709421 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:06.709390 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbe24769-cda3-4f3a-a912-b13763402dd9-bundle" (OuterVolumeSpecName: "bundle") pod "fbe24769-cda3-4f3a-a912-b13763402dd9" (UID: "fbe24769-cda3-4f3a-a912-b13763402dd9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:26:06.710902 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:06.710878 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbe24769-cda3-4f3a-a912-b13763402dd9-kube-api-access-zlb6c" (OuterVolumeSpecName: "kube-api-access-zlb6c") pod "fbe24769-cda3-4f3a-a912-b13763402dd9" (UID: "fbe24769-cda3-4f3a-a912-b13763402dd9"). InnerVolumeSpecName "kube-api-access-zlb6c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:26:06.714327 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:06.714306 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbe24769-cda3-4f3a-a912-b13763402dd9-util" (OuterVolumeSpecName: "util") pod "fbe24769-cda3-4f3a-a912-b13763402dd9" (UID: "fbe24769-cda3-4f3a-a912-b13763402dd9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:26:06.810393 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:06.810367 2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fbe24769-cda3-4f3a-a912-b13763402dd9-bundle\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:26:06.810393 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:06.810390 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlb6c\" (UniqueName: \"kubernetes.io/projected/fbe24769-cda3-4f3a-a912-b13763402dd9-kube-api-access-zlb6c\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:26:06.810539 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:06.810399 2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fbe24769-cda3-4f3a-a912-b13763402dd9-util\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:26:07.462884 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:07.462847 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp" event={"ID":"fbe24769-cda3-4f3a-a912-b13763402dd9","Type":"ContainerDied","Data":"0ea24a0708cca8e0880b34895556408aa98ee2f55c7e7e3342ff77f52a66e7f9"} Jul 14 11:26:07.463292 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:07.462892 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ea24a0708cca8e0880b34895556408aa98ee2f55c7e7e3342ff77f52a66e7f9" Jul 14 11:26:07.463292 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:07.462858 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp" Jul 14 11:26:11.367817 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:11.367784 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:11.426056 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:11.426023 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:11.618864 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:11.618793 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:11.618864 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:11.618849 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:11.623573 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:11.623544 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:12.490697 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:12.490666 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79dbc94c75-fvgsh" Jul 14 11:26:12.542723 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:12.542690 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d655fdd4b-h7zqq"] Jul 14 11:26:12.657919 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:12.657875 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:12.760447 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:12.760368 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:12.822190 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:12.822154 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:13.807027 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:13.806988 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:17.260232 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:17.260198 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:19.210074 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:19.210035 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:20.799367 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:20.799336 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:21.206579 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:21.206546 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:22.648052 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:22.648024 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:24.244814 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:24.244780 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:25.154836 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:25.154784 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:25.305728 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:25.305697 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:27.059808 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:27.059775 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:27.638180 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:27.638130 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:28.283315 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:28.283283 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:29.025859 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:29.025825 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:29.125810 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:29.125767 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:35.795940 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:35.795901 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:35.899026 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:35.898990 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:36.062737 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:36.062648 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:36.198511 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:36.198479 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:37.578345 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.578291 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d655fdd4b-h7zqq" podUID="6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77" containerName="console" containerID="cri-o://dfadc71a82b8671e47612f354115648e6760c301402f5f0a618966eaa7adb15f" gracePeriod=15 Jul 14 11:26:37.814074 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.814047 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:26:37.881111 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.881084 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-oauth-serving-cert\") pod \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " Jul 14 11:26:37.881917 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.881137 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-console-serving-cert\") pod \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " Jul 14 11:26:37.881917 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.881163 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-console-oauth-config\") pod \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " Jul 14 11:26:37.881917 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.881196 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-trusted-ca-bundle\") pod \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " Jul 14 11:26:37.881917 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.881231 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8bnj\" (UniqueName: \"kubernetes.io/projected/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-kube-api-access-p8bnj\") pod \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " Jul 14 11:26:37.881917 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.881278 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-service-ca\") pod \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " Jul 14 11:26:37.881917 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.881785 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-service-ca" (OuterVolumeSpecName: "service-ca") pod "6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77" (UID: "6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:26:37.881917 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.881830 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77" (UID: "6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:26:37.881917 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.881861 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77" (UID: "6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:26:37.881917 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.881891 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-console-config\") pod \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\" (UID: \"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77\") " Jul 14 11:26:37.882356 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.882205 2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-oauth-serving-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:26:37.882356 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.882224 2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-trusted-ca-bundle\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:26:37.882356 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.882237 2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-service-ca\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:26:37.882356 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.882314 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-console-config" (OuterVolumeSpecName: "console-config") pod "6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77" (UID: "6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:26:37.884048 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.884004 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-kube-api-access-p8bnj" (OuterVolumeSpecName: "kube-api-access-p8bnj") pod "6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77" (UID: "6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77"). InnerVolumeSpecName "kube-api-access-p8bnj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:26:37.884275 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.884251 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77" (UID: "6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:26:37.884362 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.884283 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77" (UID: "6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:26:37.983571 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.983537 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8bnj\" (UniqueName: \"kubernetes.io/projected/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-kube-api-access-p8bnj\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:26:37.983571 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.983567 2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-console-config\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:26:37.983571 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.983577 2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-console-serving-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:26:37.983794 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:37.983585 2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77-console-oauth-config\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:26:38.617808 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:38.617769 2571 generic.go:356] "Generic (PLEG): container finished" podID="6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77" containerID="dfadc71a82b8671e47612f354115648e6760c301402f5f0a618966eaa7adb15f" exitCode=0 Jul 14 11:26:38.618194 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:38.617819 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d655fdd4b-h7zqq" Jul 14 11:26:38.618194 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:38.617855 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d655fdd4b-h7zqq" event={"ID":"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77","Type":"ContainerDied","Data":"dfadc71a82b8671e47612f354115648e6760c301402f5f0a618966eaa7adb15f"} Jul 14 11:26:38.618194 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:38.617906 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d655fdd4b-h7zqq" event={"ID":"6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77","Type":"ContainerDied","Data":"cb7020a8f3090110b10e5cdab73079146d42dbda11568df510edfbe09146821b"} Jul 14 11:26:38.618194 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:38.617921 2571 scope.go:117] "RemoveContainer" containerID="dfadc71a82b8671e47612f354115648e6760c301402f5f0a618966eaa7adb15f" Jul 14 11:26:38.626317 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:38.626300 2571 scope.go:117] "RemoveContainer" containerID="dfadc71a82b8671e47612f354115648e6760c301402f5f0a618966eaa7adb15f" Jul 14 11:26:38.626614 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:26:38.626595 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfadc71a82b8671e47612f354115648e6760c301402f5f0a618966eaa7adb15f\": container with ID starting with dfadc71a82b8671e47612f354115648e6760c301402f5f0a618966eaa7adb15f not found: ID does not exist" containerID="dfadc71a82b8671e47612f354115648e6760c301402f5f0a618966eaa7adb15f" Jul 14 11:26:38.626706 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:38.626622 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfadc71a82b8671e47612f354115648e6760c301402f5f0a618966eaa7adb15f"} err="failed to get container status \"dfadc71a82b8671e47612f354115648e6760c301402f5f0a618966eaa7adb15f\": rpc error: code = NotFound desc = could not find container \"dfadc71a82b8671e47612f354115648e6760c301402f5f0a618966eaa7adb15f\": container with ID starting with dfadc71a82b8671e47612f354115648e6760c301402f5f0a618966eaa7adb15f not found: ID does not exist" Jul 14 11:26:38.667892 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:38.667820 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d655fdd4b-h7zqq"] Jul 14 11:26:38.688815 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:38.688788 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d655fdd4b-h7zqq"] Jul 14 11:26:40.403607 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:40.403575 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77" path="/var/lib/kubelet/pods/6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77/volumes" Jul 14 11:26:40.964037 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:40.963999 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:41.095998 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:41.095947 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:43.349608 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:43.349565 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:43.448902 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:43.448869 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:48.863842 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:48.863806 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:48.963978 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:48.963939 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:55.903378 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:55.903321 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:26:56.003439 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:26:56.003409 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:00.605219 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:00.605174 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:00.705887 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:00.705854 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:03.061286 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:03.061244 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:03.149776 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:03.149740 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:08.583769 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.583736 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr"] Jul 14 11:27:08.584238 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584203 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc697cb2-85ad-4af1-b87f-7f30203b1bd2" containerName="util" Jul 14 11:27:08.584238 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584224 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc697cb2-85ad-4af1-b87f-7f30203b1bd2" containerName="util" Jul 14 11:27:08.584350 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584243 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef5afc8c-1590-43b6-b740-5768dd559481" containerName="util" Jul 14 11:27:08.584350 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584254 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef5afc8c-1590-43b6-b740-5768dd559481" containerName="util" Jul 14 11:27:08.584350 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584264 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5685eb77-c028-43e8-8532-20e60bf76588" containerName="pull" Jul 14 11:27:08.584350 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584273 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5685eb77-c028-43e8-8532-20e60bf76588" containerName="pull" Jul 14 11:27:08.584350 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584287 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef5afc8c-1590-43b6-b740-5768dd559481" containerName="extract" Jul 14 11:27:08.584350 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584295 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef5afc8c-1590-43b6-b740-5768dd559481" containerName="extract" Jul 14 11:27:08.584350 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584306 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbe24769-cda3-4f3a-a912-b13763402dd9" containerName="util" Jul 14 11:27:08.584350 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584315 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbe24769-cda3-4f3a-a912-b13763402dd9" containerName="util" Jul 14 11:27:08.584350 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584329 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc697cb2-85ad-4af1-b87f-7f30203b1bd2" containerName="extract" Jul 14 11:27:08.584350 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584338 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc697cb2-85ad-4af1-b87f-7f30203b1bd2" containerName="extract" Jul 14 11:27:08.584350 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584347 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbe24769-cda3-4f3a-a912-b13763402dd9" containerName="pull" Jul 14 11:27:08.584350 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584354 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbe24769-cda3-4f3a-a912-b13763402dd9" containerName="pull" Jul 14 11:27:08.584822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584369 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc697cb2-85ad-4af1-b87f-7f30203b1bd2" containerName="pull" Jul 14 11:27:08.584822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584379 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc697cb2-85ad-4af1-b87f-7f30203b1bd2" containerName="pull" Jul 14 11:27:08.584822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584397 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77" containerName="console" Jul 14 11:27:08.584822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584406 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77" containerName="console" Jul 14 11:27:08.584822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584416 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5685eb77-c028-43e8-8532-20e60bf76588" containerName="extract" Jul 14 11:27:08.584822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584425 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5685eb77-c028-43e8-8532-20e60bf76588" containerName="extract" Jul 14 11:27:08.584822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584439 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5685eb77-c028-43e8-8532-20e60bf76588" containerName="util" Jul 14 11:27:08.584822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584449 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5685eb77-c028-43e8-8532-20e60bf76588" containerName="util" Jul 14 11:27:08.584822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584459 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef5afc8c-1590-43b6-b740-5768dd559481" containerName="pull" Jul 14 11:27:08.584822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584466 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef5afc8c-1590-43b6-b740-5768dd559481" containerName="pull" Jul 14 11:27:08.584822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584476 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbe24769-cda3-4f3a-a912-b13763402dd9" containerName="extract" Jul 14 11:27:08.584822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584484 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbe24769-cda3-4f3a-a912-b13763402dd9" containerName="extract" Jul 14 11:27:08.584822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584569 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c43df2b-54b8-4329-8fb7-4ab2ae4ebf77" containerName="console" Jul 14 11:27:08.584822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584582 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef5afc8c-1590-43b6-b740-5768dd559481" containerName="extract" Jul 14 11:27:08.584822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584595 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbe24769-cda3-4f3a-a912-b13763402dd9" containerName="extract" Jul 14 11:27:08.584822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584604 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5685eb77-c028-43e8-8532-20e60bf76588" containerName="extract" Jul 14 11:27:08.584822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.584614 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc697cb2-85ad-4af1-b87f-7f30203b1bd2" containerName="extract" Jul 14 11:27:08.587314 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.587299 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.590239 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.590218 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-vwfjt\"" Jul 14 11:27:08.600239 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.600216 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr"] Jul 14 11:27:08.612344 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.612316 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:08.731292 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.731262 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a4abd65-ce9d-4eec-a797-f4762b7538aa-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.731292 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.731293 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a4abd65-ce9d-4eec-a797-f4762b7538aa-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.731464 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.731331 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a4abd65-ce9d-4eec-a797-f4762b7538aa-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.731464 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.731373 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a4abd65-ce9d-4eec-a797-f4762b7538aa-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.731464 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.731391 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a4abd65-ce9d-4eec-a797-f4762b7538aa-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.731589 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.731469 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a4abd65-ce9d-4eec-a797-f4762b7538aa-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.731589 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.731501 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a4abd65-ce9d-4eec-a797-f4762b7538aa-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.731589 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.731518 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rjkt\" (UniqueName: \"kubernetes.io/projected/6a4abd65-ce9d-4eec-a797-f4762b7538aa-kube-api-access-8rjkt\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.731736 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.731590 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a4abd65-ce9d-4eec-a797-f4762b7538aa-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.818295 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.818266 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:08.832945 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.832921 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a4abd65-ce9d-4eec-a797-f4762b7538aa-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.833089 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.832960 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a4abd65-ce9d-4eec-a797-f4762b7538aa-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.833089 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.832985 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a4abd65-ce9d-4eec-a797-f4762b7538aa-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.833089 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.833030 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a4abd65-ce9d-4eec-a797-f4762b7538aa-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.833089 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.833068 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a4abd65-ce9d-4eec-a797-f4762b7538aa-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.833298 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.833093 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rjkt\" (UniqueName: \"kubernetes.io/projected/6a4abd65-ce9d-4eec-a797-f4762b7538aa-kube-api-access-8rjkt\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.833298 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.833219 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a4abd65-ce9d-4eec-a797-f4762b7538aa-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.833298 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.833283 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a4abd65-ce9d-4eec-a797-f4762b7538aa-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.833451 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.833315 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a4abd65-ce9d-4eec-a797-f4762b7538aa-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.833451 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.833385 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a4abd65-ce9d-4eec-a797-f4762b7538aa-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.833562 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.833507 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a4abd65-ce9d-4eec-a797-f4762b7538aa-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.833562 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.833507 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a4abd65-ce9d-4eec-a797-f4762b7538aa-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.833736 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.833712 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a4abd65-ce9d-4eec-a797-f4762b7538aa-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.833810 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.833772 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a4abd65-ce9d-4eec-a797-f4762b7538aa-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.835415 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.835396 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a4abd65-ce9d-4eec-a797-f4762b7538aa-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.835529 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.835513 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a4abd65-ce9d-4eec-a797-f4762b7538aa-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.842351 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.842306 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rjkt\" (UniqueName: \"kubernetes.io/projected/6a4abd65-ce9d-4eec-a797-f4762b7538aa-kube-api-access-8rjkt\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.842808 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.842787 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a4abd65-ce9d-4eec-a797-f4762b7538aa-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-llkzr\" (UID: \"6a4abd65-ce9d-4eec-a797-f4762b7538aa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.897612 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.897582 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:08.927414 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:08.927384 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:09.033244 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:09.033219 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr"] Jul 14 11:27:09.034982 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:27:09.034947 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a4abd65_ce9d_4eec_a797_f4762b7538aa.slice/crio-fa65ccb2c26c4133e550de0b10a5deabaa929c60153401035759ae84517da880 WatchSource:0}: Error finding container fa65ccb2c26c4133e550de0b10a5deabaa929c60153401035759ae84517da880: Status 404 returned error can't find the container with id fa65ccb2c26c4133e550de0b10a5deabaa929c60153401035759ae84517da880 Jul 14 11:27:09.037018 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:09.036990 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:27:09.037091 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:09.037066 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:27:09.037138 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:09.037100 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:27:09.768369 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:09.768335 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" event={"ID":"6a4abd65-ce9d-4eec-a797-f4762b7538aa","Type":"ContainerStarted","Data":"cc1800f79f6a29c14374ff8db473656be9495fbbe75b3bc4789cae43740c2038"} Jul 14 11:27:09.768856 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:09.768388 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" event={"ID":"6a4abd65-ce9d-4eec-a797-f4762b7538aa","Type":"ContainerStarted","Data":"fa65ccb2c26c4133e550de0b10a5deabaa929c60153401035759ae84517da880"} Jul 14 11:27:09.794098 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:09.794046 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" podStartSLOduration=1.794025645 podStartE2EDuration="1.794025645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:27:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:27:09.792105193 +0000 UTC m=+523.953181669" watchObservedRunningTime="2026-07-14 11:27:09.794025645 +0000 UTC m=+523.955102113" Jul 14 11:27:09.898712 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:09.898684 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:10.895778 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:10.895736 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:10.903146 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:10.903123 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:11.012898 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:11.012865 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:11.071970 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:11.071941 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:11.736849 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:11.736813 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:11.777795 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:11.777768 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:11.778686 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:11.778670 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:27:11.808423 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:11.808395 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:16.616027 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:16.615975 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:16.717435 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:16.717385 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:19.021872 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:19.021792 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:21.015776 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:21.015745 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:21.116581 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:21.116547 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:23.465474 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:23.465438 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:23.565311 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:23.565286 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:27.190134 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:27.190098 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:27.281419 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:27.281384 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:27.881994 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:27.881953 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:39.677472 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:39.677439 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:39.931756 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:39.931490 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:47.293215 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.293173 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:47.364957 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.364922 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6dbbf9ccd8-xhc8q"] Jul 14 11:27:47.369703 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.369684 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dbbf9ccd8-xhc8q" Jul 14 11:27:47.372732 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.372714 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 14 11:27:47.372831 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.372813 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4rv5q\"" Jul 14 11:27:47.373852 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.373831 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 14 11:27:47.380847 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.380827 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6dbbf9ccd8-xhc8q"] Jul 14 11:27:47.405745 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.405723 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:47.440099 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.440071 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm7vc\" (UniqueName: \"kubernetes.io/projected/fa82d28f-56d0-4a65-954b-e922ea3817e8-kube-api-access-wm7vc\") pod \"maas-controller-6dbbf9ccd8-xhc8q\" (UID: \"fa82d28f-56d0-4a65-954b-e922ea3817e8\") " pod="opendatahub/maas-controller-6dbbf9ccd8-xhc8q" Jul 14 11:27:47.440208 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.440174 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa82d28f-56d0-4a65-954b-e922ea3817e8-webhook-cert\") pod \"maas-controller-6dbbf9ccd8-xhc8q\" (UID: \"fa82d28f-56d0-4a65-954b-e922ea3817e8\") " pod="opendatahub/maas-controller-6dbbf9ccd8-xhc8q" Jul 14 11:27:47.472752 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.472728 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:47.502874 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.502849 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f86cb7fff-npfxr"] Jul 14 11:27:47.506255 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.506233 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f86cb7fff-npfxr" Jul 14 11:27:47.520603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.520576 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f86cb7fff-npfxr"] Jul 14 11:27:47.540644 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.540608 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa82d28f-56d0-4a65-954b-e922ea3817e8-webhook-cert\") pod \"maas-controller-6dbbf9ccd8-xhc8q\" (UID: \"fa82d28f-56d0-4a65-954b-e922ea3817e8\") " pod="opendatahub/maas-controller-6dbbf9ccd8-xhc8q" Jul 14 11:27:47.540736 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.540680 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de2bc2b1-aaaf-4794-84f8-ccf53c646dd3-webhook-cert\") pod \"maas-controller-5f86cb7fff-npfxr\" (UID: \"de2bc2b1-aaaf-4794-84f8-ccf53c646dd3\") " pod="opendatahub/maas-controller-5f86cb7fff-npfxr" Jul 14 11:27:47.540736 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.540705 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm7vc\" (UniqueName: \"kubernetes.io/projected/fa82d28f-56d0-4a65-954b-e922ea3817e8-kube-api-access-wm7vc\") pod \"maas-controller-6dbbf9ccd8-xhc8q\" (UID: \"fa82d28f-56d0-4a65-954b-e922ea3817e8\") " pod="opendatahub/maas-controller-6dbbf9ccd8-xhc8q" Jul 14 11:27:47.540817 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.540785 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl86n\" (UniqueName: \"kubernetes.io/projected/de2bc2b1-aaaf-4794-84f8-ccf53c646dd3-kube-api-access-vl86n\") pod \"maas-controller-5f86cb7fff-npfxr\" (UID: \"de2bc2b1-aaaf-4794-84f8-ccf53c646dd3\") " pod="opendatahub/maas-controller-5f86cb7fff-npfxr" Jul 14 11:27:47.542950 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.542934 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa82d28f-56d0-4a65-954b-e922ea3817e8-webhook-cert\") pod \"maas-controller-6dbbf9ccd8-xhc8q\" (UID: \"fa82d28f-56d0-4a65-954b-e922ea3817e8\") " pod="opendatahub/maas-controller-6dbbf9ccd8-xhc8q" Jul 14 11:27:47.551279 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.551231 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm7vc\" (UniqueName: \"kubernetes.io/projected/fa82d28f-56d0-4a65-954b-e922ea3817e8-kube-api-access-wm7vc\") pod \"maas-controller-6dbbf9ccd8-xhc8q\" (UID: \"fa82d28f-56d0-4a65-954b-e922ea3817e8\") " pod="opendatahub/maas-controller-6dbbf9ccd8-xhc8q" Jul 14 11:27:47.570390 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.570364 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:47.641912 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.641886 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl86n\" (UniqueName: \"kubernetes.io/projected/de2bc2b1-aaaf-4794-84f8-ccf53c646dd3-kube-api-access-vl86n\") pod \"maas-controller-5f86cb7fff-npfxr\" (UID: \"de2bc2b1-aaaf-4794-84f8-ccf53c646dd3\") " pod="opendatahub/maas-controller-5f86cb7fff-npfxr" Jul 14 11:27:47.642010 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.641978 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de2bc2b1-aaaf-4794-84f8-ccf53c646dd3-webhook-cert\") pod \"maas-controller-5f86cb7fff-npfxr\" (UID: \"de2bc2b1-aaaf-4794-84f8-ccf53c646dd3\") " pod="opendatahub/maas-controller-5f86cb7fff-npfxr" Jul 14 11:27:47.644125 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.644108 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de2bc2b1-aaaf-4794-84f8-ccf53c646dd3-webhook-cert\") pod \"maas-controller-5f86cb7fff-npfxr\" (UID: \"de2bc2b1-aaaf-4794-84f8-ccf53c646dd3\") " pod="opendatahub/maas-controller-5f86cb7fff-npfxr" Jul 14 11:27:47.651545 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.651526 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl86n\" (UniqueName: \"kubernetes.io/projected/de2bc2b1-aaaf-4794-84f8-ccf53c646dd3-kube-api-access-vl86n\") pod \"maas-controller-5f86cb7fff-npfxr\" (UID: \"de2bc2b1-aaaf-4794-84f8-ccf53c646dd3\") " pod="opendatahub/maas-controller-5f86cb7fff-npfxr" Jul 14 11:27:47.680388 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.680354 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dbbf9ccd8-xhc8q" Jul 14 11:27:47.811587 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.811559 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6dbbf9ccd8-xhc8q"] Jul 14 11:27:47.812493 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:27:47.812467 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa82d28f_56d0_4a65_954b_e922ea3817e8.slice/crio-9edf21a78c9c267c2300b833c8c5fc7e08182644b44b6bf6e25ac39c85d7f60a WatchSource:0}: Error finding container 9edf21a78c9c267c2300b833c8c5fc7e08182644b44b6bf6e25ac39c85d7f60a: Status 404 returned error can't find the container with id 9edf21a78c9c267c2300b833c8c5fc7e08182644b44b6bf6e25ac39c85d7f60a Jul 14 11:27:47.816346 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.816329 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f86cb7fff-npfxr" Jul 14 11:27:47.942646 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.942604 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f86cb7fff-npfxr"] Jul 14 11:27:47.944896 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:27:47.944868 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde2bc2b1_aaaf_4794_84f8_ccf53c646dd3.slice/crio-174f92cdcd6f0947061352058556adcdaf5e30c5be96ec714474f4924687a5c5 WatchSource:0}: Error finding container 174f92cdcd6f0947061352058556adcdaf5e30c5be96ec714474f4924687a5c5: Status 404 returned error can't find the container with id 174f92cdcd6f0947061352058556adcdaf5e30c5be96ec714474f4924687a5c5 Jul 14 11:27:47.961717 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.961682 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f86cb7fff-npfxr" event={"ID":"de2bc2b1-aaaf-4794-84f8-ccf53c646dd3","Type":"ContainerStarted","Data":"174f92cdcd6f0947061352058556adcdaf5e30c5be96ec714474f4924687a5c5"} Jul 14 11:27:47.962846 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:47.962825 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dbbf9ccd8-xhc8q" event={"ID":"fa82d28f-56d0-4a65-954b-e922ea3817e8","Type":"ContainerStarted","Data":"9edf21a78c9c267c2300b833c8c5fc7e08182644b44b6bf6e25ac39c85d7f60a"} Jul 14 11:27:49.979121 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:49.979088 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f86cb7fff-npfxr" event={"ID":"de2bc2b1-aaaf-4794-84f8-ccf53c646dd3","Type":"ContainerStarted","Data":"20489015c62f237f2fdec56cf59b0c9a260fcda95f8fd61ca6da75bd7388896c"} Jul 14 11:27:49.979577 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:49.979251 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f86cb7fff-npfxr" Jul 14 11:27:49.981064 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:49.981036 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dbbf9ccd8-xhc8q" event={"ID":"fa82d28f-56d0-4a65-954b-e922ea3817e8","Type":"ContainerStarted","Data":"8721d834828ebd282bdf92b753cb893dc1380414903c1c7b038c066c65322330"} Jul 14 11:27:49.981187 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:49.981162 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6dbbf9ccd8-xhc8q" Jul 14 11:27:49.998770 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:49.998712 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f86cb7fff-npfxr" podStartSLOduration=1.109334313 podStartE2EDuration="2.998695052s" totalImagesPullingTime="1.889360739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:27:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:27:49.997826982 +0000 UTC m=+564.158903449" watchObservedRunningTime="2026-07-14 11:27:49.998695052 +0000 UTC m=+564.159771532" Jul 14 11:27:50.029835 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:50.029780 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6dbbf9ccd8-xhc8q" podStartSLOduration=1.018065078 podStartE2EDuration="3.029763341s" totalImagesPullingTime="2.011698263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:27:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:27:50.028048125 +0000 UTC m=+564.189124584" watchObservedRunningTime="2026-07-14 11:27:50.029763341 +0000 UTC m=+564.190839810" Jul 14 11:27:50.077387 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:50.077292 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:50.158087 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:50.158058 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:50.566428 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:50.566399 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:50.627264 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:50.627232 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:57.043925 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:57.043889 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:27:57.194520 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:27:57.194488 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:28:00.987944 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:00.987915 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f86cb7fff-npfxr" Jul 14 11:28:00.988279 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:00.987981 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6dbbf9ccd8-xhc8q" Jul 14 11:28:01.065740 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:01.065709 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6dbbf9ccd8-xhc8q"] Jul 14 11:28:01.065897 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:01.065869 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6dbbf9ccd8-xhc8q" podUID="fa82d28f-56d0-4a65-954b-e922ea3817e8" containerName="manager" containerID="cri-o://8721d834828ebd282bdf92b753cb893dc1380414903c1c7b038c066c65322330" gracePeriod=30 Jul 14 11:28:01.307041 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:01.307018 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dbbf9ccd8-xhc8q" Jul 14 11:28:01.353860 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:01.353833 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa82d28f-56d0-4a65-954b-e922ea3817e8-webhook-cert\") pod \"fa82d28f-56d0-4a65-954b-e922ea3817e8\" (UID: \"fa82d28f-56d0-4a65-954b-e922ea3817e8\") " Jul 14 11:28:01.354008 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:01.353930 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wm7vc\" (UniqueName: \"kubernetes.io/projected/fa82d28f-56d0-4a65-954b-e922ea3817e8-kube-api-access-wm7vc\") pod \"fa82d28f-56d0-4a65-954b-e922ea3817e8\" (UID: \"fa82d28f-56d0-4a65-954b-e922ea3817e8\") " Jul 14 11:28:01.355876 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:01.355845 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa82d28f-56d0-4a65-954b-e922ea3817e8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "fa82d28f-56d0-4a65-954b-e922ea3817e8" (UID: "fa82d28f-56d0-4a65-954b-e922ea3817e8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:28:01.355966 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:01.355904 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa82d28f-56d0-4a65-954b-e922ea3817e8-kube-api-access-wm7vc" (OuterVolumeSpecName: "kube-api-access-wm7vc") pod "fa82d28f-56d0-4a65-954b-e922ea3817e8" (UID: "fa82d28f-56d0-4a65-954b-e922ea3817e8"). InnerVolumeSpecName "kube-api-access-wm7vc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:28:01.455087 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:01.455049 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wm7vc\" (UniqueName: \"kubernetes.io/projected/fa82d28f-56d0-4a65-954b-e922ea3817e8-kube-api-access-wm7vc\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:28:01.455087 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:01.455086 2571 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa82d28f-56d0-4a65-954b-e922ea3817e8-webhook-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:28:02.043288 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:02.043259 2571 generic.go:356] "Generic (PLEG): container finished" podID="fa82d28f-56d0-4a65-954b-e922ea3817e8" containerID="8721d834828ebd282bdf92b753cb893dc1380414903c1c7b038c066c65322330" exitCode=0 Jul 14 11:28:02.043687 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:02.043306 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dbbf9ccd8-xhc8q" Jul 14 11:28:02.043687 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:02.043342 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dbbf9ccd8-xhc8q" event={"ID":"fa82d28f-56d0-4a65-954b-e922ea3817e8","Type":"ContainerDied","Data":"8721d834828ebd282bdf92b753cb893dc1380414903c1c7b038c066c65322330"} Jul 14 11:28:02.043687 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:02.043391 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dbbf9ccd8-xhc8q" event={"ID":"fa82d28f-56d0-4a65-954b-e922ea3817e8","Type":"ContainerDied","Data":"9edf21a78c9c267c2300b833c8c5fc7e08182644b44b6bf6e25ac39c85d7f60a"} Jul 14 11:28:02.043687 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:02.043415 2571 scope.go:117] "RemoveContainer" containerID="8721d834828ebd282bdf92b753cb893dc1380414903c1c7b038c066c65322330" Jul 14 11:28:02.053983 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:02.053963 2571 scope.go:117] "RemoveContainer" containerID="8721d834828ebd282bdf92b753cb893dc1380414903c1c7b038c066c65322330" Jul 14 11:28:02.054392 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:28:02.054371 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8721d834828ebd282bdf92b753cb893dc1380414903c1c7b038c066c65322330\": container with ID starting with 8721d834828ebd282bdf92b753cb893dc1380414903c1c7b038c066c65322330 not found: ID does not exist" containerID="8721d834828ebd282bdf92b753cb893dc1380414903c1c7b038c066c65322330" Jul 14 11:28:02.054473 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:02.054402 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8721d834828ebd282bdf92b753cb893dc1380414903c1c7b038c066c65322330"} err="failed to get container status \"8721d834828ebd282bdf92b753cb893dc1380414903c1c7b038c066c65322330\": rpc error: code = NotFound desc = could not find container \"8721d834828ebd282bdf92b753cb893dc1380414903c1c7b038c066c65322330\": container with ID starting with 8721d834828ebd282bdf92b753cb893dc1380414903c1c7b038c066c65322330 not found: ID does not exist" Jul 14 11:28:02.064915 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:02.064895 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6dbbf9ccd8-xhc8q"] Jul 14 11:28:02.070896 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:02.070877 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6dbbf9ccd8-xhc8q"] Jul 14 11:28:02.403208 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:02.403179 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa82d28f-56d0-4a65-954b-e922ea3817e8" path="/var/lib/kubelet/pods/fa82d28f-56d0-4a65-954b-e922ea3817e8/volumes" Jul 14 11:28:08.782913 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:08.782881 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:28:18.761759 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:18.761716 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:28:18.867186 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:18.867154 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:28:19.191682 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.191647 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:28:19.311296 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.311260 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-68f88d65bd-gkcsv"] Jul 14 11:28:19.311648 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.311618 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa82d28f-56d0-4a65-954b-e922ea3817e8" containerName="manager" Jul 14 11:28:19.311700 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.311652 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa82d28f-56d0-4a65-954b-e922ea3817e8" containerName="manager" Jul 14 11:28:19.311742 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.311733 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa82d28f-56d0-4a65-954b-e922ea3817e8" containerName="manager" Jul 14 11:28:19.316156 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.316137 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68f88d65bd-gkcsv" Jul 14 11:28:19.319127 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.319109 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5pg7p\"" Jul 14 11:28:19.319357 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.319337 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 11:28:19.319438 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.319345 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 14 11:28:19.320454 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.320435 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 14 11:28:19.328606 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.328585 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68f88d65bd-gkcsv"] Jul 14 11:28:19.391362 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.391332 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mnll\" (UniqueName: \"kubernetes.io/projected/d6c90d15-7994-417e-8553-4631640569ea-kube-api-access-4mnll\") pod \"maas-api-68f88d65bd-gkcsv\" (UID: \"d6c90d15-7994-417e-8553-4631640569ea\") " pod="odh-ai-gateway-infra/maas-api-68f88d65bd-gkcsv" Jul 14 11:28:19.391362 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.391365 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6c90d15-7994-417e-8553-4631640569ea-openshift-service-ca\") pod \"maas-api-68f88d65bd-gkcsv\" (UID: \"d6c90d15-7994-417e-8553-4631640569ea\") " pod="odh-ai-gateway-infra/maas-api-68f88d65bd-gkcsv" Jul 14 11:28:19.391547 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.391383 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6c90d15-7994-417e-8553-4631640569ea-maas-api-tls\") pod \"maas-api-68f88d65bd-gkcsv\" (UID: \"d6c90d15-7994-417e-8553-4631640569ea\") " pod="odh-ai-gateway-infra/maas-api-68f88d65bd-gkcsv" Jul 14 11:28:19.492504 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.492444 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mnll\" (UniqueName: \"kubernetes.io/projected/d6c90d15-7994-417e-8553-4631640569ea-kube-api-access-4mnll\") pod \"maas-api-68f88d65bd-gkcsv\" (UID: \"d6c90d15-7994-417e-8553-4631640569ea\") " pod="odh-ai-gateway-infra/maas-api-68f88d65bd-gkcsv" Jul 14 11:28:19.492504 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.492474 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6c90d15-7994-417e-8553-4631640569ea-openshift-service-ca\") pod \"maas-api-68f88d65bd-gkcsv\" (UID: \"d6c90d15-7994-417e-8553-4631640569ea\") " pod="odh-ai-gateway-infra/maas-api-68f88d65bd-gkcsv" Jul 14 11:28:19.492504 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.492494 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6c90d15-7994-417e-8553-4631640569ea-maas-api-tls\") pod \"maas-api-68f88d65bd-gkcsv\" (UID: \"d6c90d15-7994-417e-8553-4631640569ea\") " pod="odh-ai-gateway-infra/maas-api-68f88d65bd-gkcsv" Jul 14 11:28:19.492744 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:28:19.492606 2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 14 11:28:19.492744 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:28:19.492699 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6c90d15-7994-417e-8553-4631640569ea-maas-api-tls podName:d6c90d15-7994-417e-8553-4631640569ea nodeName:}" failed. No retries permitted until 2026-07-14 11:28:19.992679023 +0000 UTC m=+594.153755470 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d6c90d15-7994-417e-8553-4631640569ea-maas-api-tls") pod "maas-api-68f88d65bd-gkcsv" (UID: "d6c90d15-7994-417e-8553-4631640569ea") : secret "maas-api-serving-cert" not found Jul 14 11:28:19.493112 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.493093 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6c90d15-7994-417e-8553-4631640569ea-openshift-service-ca\") pod \"maas-api-68f88d65bd-gkcsv\" (UID: \"d6c90d15-7994-417e-8553-4631640569ea\") " pod="odh-ai-gateway-infra/maas-api-68f88d65bd-gkcsv" Jul 14 11:28:19.503241 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.503214 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mnll\" (UniqueName: \"kubernetes.io/projected/d6c90d15-7994-417e-8553-4631640569ea-kube-api-access-4mnll\") pod \"maas-api-68f88d65bd-gkcsv\" (UID: \"d6c90d15-7994-417e-8553-4631640569ea\") " pod="odh-ai-gateway-infra/maas-api-68f88d65bd-gkcsv" Jul 14 11:28:19.702649 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.702604 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:28:19.843384 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.843300 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:28:19.943697 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.943654 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:28:19.996423 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.996401 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6c90d15-7994-417e-8553-4631640569ea-maas-api-tls\") pod \"maas-api-68f88d65bd-gkcsv\" (UID: \"d6c90d15-7994-417e-8553-4631640569ea\") " pod="odh-ai-gateway-infra/maas-api-68f88d65bd-gkcsv" Jul 14 11:28:19.998917 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:19.998891 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6c90d15-7994-417e-8553-4631640569ea-maas-api-tls\") pod \"maas-api-68f88d65bd-gkcsv\" (UID: \"d6c90d15-7994-417e-8553-4631640569ea\") " pod="odh-ai-gateway-infra/maas-api-68f88d65bd-gkcsv" Jul 14 11:28:20.226930 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:20.226846 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68f88d65bd-gkcsv" Jul 14 11:28:20.363370 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:20.363347 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68f88d65bd-gkcsv"] Jul 14 11:28:20.365138 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:28:20.365111 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6c90d15_7994_417e_8553_4631640569ea.slice/crio-48358597c4fa5e056ffe2922744fd810bd1abda18c580dc2f385d4de0de5b87c WatchSource:0}: Error finding container 48358597c4fa5e056ffe2922744fd810bd1abda18c580dc2f385d4de0de5b87c: Status 404 returned error can't find the container with id 48358597c4fa5e056ffe2922744fd810bd1abda18c580dc2f385d4de0de5b87c Jul 14 11:28:21.149957 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:21.149918 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68f88d65bd-gkcsv" event={"ID":"d6c90d15-7994-417e-8553-4631640569ea","Type":"ContainerStarted","Data":"48358597c4fa5e056ffe2922744fd810bd1abda18c580dc2f385d4de0de5b87c"} Jul 14 11:28:22.156016 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:22.155986 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68f88d65bd-gkcsv" event={"ID":"d6c90d15-7994-417e-8553-4631640569ea","Type":"ContainerStarted","Data":"b94cd627d9ac2fd1c8c0f80af03efc77283e176d0940606b848515f52b1b1752"} Jul 14 11:28:22.156384 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:22.156068 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-68f88d65bd-gkcsv" Jul 14 11:28:22.178645 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:22.175809 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-68f88d65bd-gkcsv" podStartSLOduration=1.549496114 podStartE2EDuration="3.17578978s" totalImagesPullingTime="1.626293666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:28:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:28:22.174390587 +0000 UTC m=+596.335467056" watchObservedRunningTime="2026-07-14 11:28:22.17578978 +0000 UTC m=+596.336866248" Jul 14 11:28:26.305612 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:26.305588 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 11:28:26.306018 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:26.305990 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 11:28:27.533849 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:27.533491 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-llkzr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": read tcp 10.133.0.2:41586->10.133.0.37:15021: read: connection reset by peer" start-of-body= Jul 14 11:28:27.533849 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:27.533551 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" podUID="6a4abd65-ce9d-4eec-a797-f4762b7538aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": read tcp 10.133.0.2:41586->10.133.0.37:15021: read: connection reset by peer" Jul 14 11:28:28.166302 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:28.166275 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-68f88d65bd-gkcsv" Jul 14 11:28:28.190430 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:28.190405 2571 generic.go:356] "Generic (PLEG): container finished" podID="6a4abd65-ce9d-4eec-a797-f4762b7538aa" containerID="cc1800f79f6a29c14374ff8db473656be9495fbbe75b3bc4789cae43740c2038" exitCode=0 Jul 14 11:28:28.190567 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:28.190452 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" event={"ID":"6a4abd65-ce9d-4eec-a797-f4762b7538aa","Type":"ContainerDied","Data":"cc1800f79f6a29c14374ff8db473656be9495fbbe75b3bc4789cae43740c2038"} Jul 14 11:28:28.190773 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:28.190757 2571 scope.go:117] "RemoveContainer" containerID="cc1800f79f6a29c14374ff8db473656be9495fbbe75b3bc4789cae43740c2038" Jul 14 11:28:28.191730 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:28.191706 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:28:28.191797 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:28.191765 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:28:28.191835 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:28.191796 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:28:29.188370 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:29.188324 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:28:29.199716 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:29.199679 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" event={"ID":"6a4abd65-ce9d-4eec-a797-f4762b7538aa","Type":"ContainerStarted","Data":"6dfa2b5f6fb6c624dd114a301f2c5c733f8000d5e34a8d25708e79d8f17b8990"} Jul 14 11:28:29.268757 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:29.268668 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:28:29.898403 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:29.898373 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:28:29.898575 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:29.898532 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-llkzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 14 11:28:29.898621 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:29.898599 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" podUID="6a4abd65-ce9d-4eec-a797-f4762b7538aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 14 11:28:30.898352 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:30.898319 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-llkzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 14 11:28:30.898732 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:30.898372 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" podUID="6a4abd65-ce9d-4eec-a797-f4762b7538aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 14 11:28:31.169208 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:31.169099 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:28:31.898997 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:31.898963 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-llkzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 14 11:28:31.899397 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:31.899016 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" podUID="6a4abd65-ce9d-4eec-a797-f4762b7538aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 14 11:28:32.898962 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:32.898924 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-llkzr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 14 11:28:32.899122 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:32.898983 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" podUID="6a4abd65-ce9d-4eec-a797-f4762b7538aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 14 11:28:33.901373 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:33.901342 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:28:33.901747 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:33.901723 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:28:33.902447 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:33.902426 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-llkzr" Jul 14 11:28:36.043596 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:36.043557 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:28:55.303962 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:28:55.303868 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:29:01.100143 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:01.100101 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:29:12.171917 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.171879 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7"] Jul 14 11:29:12.176240 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.176221 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:12.179263 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.179239 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 14 11:29:12.179367 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.179281 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 14 11:29:12.180607 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.180588 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vkrbv\"" Jul 14 11:29:12.180733 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.180717 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 14 11:29:12.186175 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.186152 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7"] Jul 14 11:29:12.268905 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.268842 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:29:12.287110 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.287085 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3acafd6-f847-428b-a64f-dd4512bab0c1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j4tj7\" (UID: \"c3acafd6-f847-428b-a64f-dd4512bab0c1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:12.287218 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.287113 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3acafd6-f847-428b-a64f-dd4512bab0c1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j4tj7\" (UID: \"c3acafd6-f847-428b-a64f-dd4512bab0c1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:12.287218 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.287150 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q95rv\" (UniqueName: \"kubernetes.io/projected/c3acafd6-f847-428b-a64f-dd4512bab0c1-kube-api-access-q95rv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j4tj7\" (UID: \"c3acafd6-f847-428b-a64f-dd4512bab0c1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:12.287218 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.287175 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3acafd6-f847-428b-a64f-dd4512bab0c1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j4tj7\" (UID: \"c3acafd6-f847-428b-a64f-dd4512bab0c1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:12.287218 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.287218 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3acafd6-f847-428b-a64f-dd4512bab0c1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j4tj7\" (UID: \"c3acafd6-f847-428b-a64f-dd4512bab0c1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:12.287360 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.287240 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3acafd6-f847-428b-a64f-dd4512bab0c1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j4tj7\" (UID: \"c3acafd6-f847-428b-a64f-dd4512bab0c1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:12.388439 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.388408 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3acafd6-f847-428b-a64f-dd4512bab0c1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j4tj7\" (UID: \"c3acafd6-f847-428b-a64f-dd4512bab0c1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:12.388439 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.388440 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3acafd6-f847-428b-a64f-dd4512bab0c1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j4tj7\" (UID: \"c3acafd6-f847-428b-a64f-dd4512bab0c1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:12.388673 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.388483 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3acafd6-f847-428b-a64f-dd4512bab0c1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j4tj7\" (UID: \"c3acafd6-f847-428b-a64f-dd4512bab0c1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:12.388673 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.388499 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3acafd6-f847-428b-a64f-dd4512bab0c1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j4tj7\" (UID: \"c3acafd6-f847-428b-a64f-dd4512bab0c1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:12.388673 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.388539 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q95rv\" (UniqueName: \"kubernetes.io/projected/c3acafd6-f847-428b-a64f-dd4512bab0c1-kube-api-access-q95rv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j4tj7\" (UID: \"c3acafd6-f847-428b-a64f-dd4512bab0c1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:12.388673 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.388558 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3acafd6-f847-428b-a64f-dd4512bab0c1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j4tj7\" (UID: \"c3acafd6-f847-428b-a64f-dd4512bab0c1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:12.388885 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.388854 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3acafd6-f847-428b-a64f-dd4512bab0c1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j4tj7\" (UID: \"c3acafd6-f847-428b-a64f-dd4512bab0c1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:12.388945 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.388917 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3acafd6-f847-428b-a64f-dd4512bab0c1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j4tj7\" (UID: \"c3acafd6-f847-428b-a64f-dd4512bab0c1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:12.389000 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.388959 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3acafd6-f847-428b-a64f-dd4512bab0c1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j4tj7\" (UID: \"c3acafd6-f847-428b-a64f-dd4512bab0c1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:12.390893 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.390870 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3acafd6-f847-428b-a64f-dd4512bab0c1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j4tj7\" (UID: \"c3acafd6-f847-428b-a64f-dd4512bab0c1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:12.391054 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.391038 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3acafd6-f847-428b-a64f-dd4512bab0c1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j4tj7\" (UID: \"c3acafd6-f847-428b-a64f-dd4512bab0c1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:12.397268 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.397251 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q95rv\" (UniqueName: \"kubernetes.io/projected/c3acafd6-f847-428b-a64f-dd4512bab0c1-kube-api-access-q95rv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-j4tj7\" (UID: \"c3acafd6-f847-428b-a64f-dd4512bab0c1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:12.486919 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.486868 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:12.616416 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.616387 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7"] Jul 14 11:29:12.616942 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:29:12.616916 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3acafd6_f847_428b_a64f_dd4512bab0c1.slice/crio-30d724a4c4efd3bfe2bf7b8b7715f75c27e4ce84181f175b606761326a203bc6 WatchSource:0}: Error finding container 30d724a4c4efd3bfe2bf7b8b7715f75c27e4ce84181f175b606761326a203bc6: Status 404 returned error can't find the container with id 30d724a4c4efd3bfe2bf7b8b7715f75c27e4ce84181f175b606761326a203bc6 Jul 14 11:29:12.618185 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:12.618168 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 11:29:13.432557 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:13.432515 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" event={"ID":"c3acafd6-f847-428b-a64f-dd4512bab0c1","Type":"ContainerStarted","Data":"30d724a4c4efd3bfe2bf7b8b7715f75c27e4ce84181f175b606761326a203bc6"} Jul 14 11:29:14.439120 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:14.439087 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" event={"ID":"c3acafd6-f847-428b-a64f-dd4512bab0c1","Type":"ContainerStarted","Data":"01ea06b8552defadfa415b05f2022a44d8d5036fe189549c22a8c96d108ef63f"} Jul 14 11:29:14.462352 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:14.462305 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" podStartSLOduration=1.4246451470000001 podStartE2EDuration="2.462290298s" totalImagesPullingTime="1.037645151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:29:14.460281182 +0000 UTC m=+648.621357662" watchObservedRunningTime="2026-07-14 11:29:14.462290298 +0000 UTC m=+648.623366796" Jul 14 11:29:17.651523 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.651493 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf"] Jul 14 11:29:17.655289 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.655270 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:17.658148 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.658109 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 14 11:29:17.667874 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.667850 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf"] Jul 14 11:29:17.678561 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.678535 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:29:17.733788 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.733762 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99d7dc85-274d-48da-812b-e523fffffc00-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jzclf\" (UID: \"99d7dc85-274d-48da-812b-e523fffffc00\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:17.733876 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.733791 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99d7dc85-274d-48da-812b-e523fffffc00-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jzclf\" (UID: \"99d7dc85-274d-48da-812b-e523fffffc00\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:17.733876 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.733820 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w67sp\" (UniqueName: \"kubernetes.io/projected/99d7dc85-274d-48da-812b-e523fffffc00-kube-api-access-w67sp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jzclf\" (UID: \"99d7dc85-274d-48da-812b-e523fffffc00\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:17.733953 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.733876 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99d7dc85-274d-48da-812b-e523fffffc00-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jzclf\" (UID: \"99d7dc85-274d-48da-812b-e523fffffc00\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:17.733953 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.733916 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99d7dc85-274d-48da-812b-e523fffffc00-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jzclf\" (UID: \"99d7dc85-274d-48da-812b-e523fffffc00\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:17.733953 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.733935 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99d7dc85-274d-48da-812b-e523fffffc00-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jzclf\" (UID: \"99d7dc85-274d-48da-812b-e523fffffc00\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:17.834976 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.834947 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99d7dc85-274d-48da-812b-e523fffffc00-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jzclf\" (UID: \"99d7dc85-274d-48da-812b-e523fffffc00\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:17.834976 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.834978 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99d7dc85-274d-48da-812b-e523fffffc00-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jzclf\" (UID: \"99d7dc85-274d-48da-812b-e523fffffc00\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:17.835176 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.835005 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w67sp\" (UniqueName: \"kubernetes.io/projected/99d7dc85-274d-48da-812b-e523fffffc00-kube-api-access-w67sp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jzclf\" (UID: \"99d7dc85-274d-48da-812b-e523fffffc00\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:17.835176 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.835029 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99d7dc85-274d-48da-812b-e523fffffc00-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jzclf\" (UID: \"99d7dc85-274d-48da-812b-e523fffffc00\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:17.835176 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.835056 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99d7dc85-274d-48da-812b-e523fffffc00-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jzclf\" (UID: \"99d7dc85-274d-48da-812b-e523fffffc00\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:17.835176 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.835081 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99d7dc85-274d-48da-812b-e523fffffc00-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jzclf\" (UID: \"99d7dc85-274d-48da-812b-e523fffffc00\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:17.835394 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.835369 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99d7dc85-274d-48da-812b-e523fffffc00-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jzclf\" (UID: \"99d7dc85-274d-48da-812b-e523fffffc00\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:17.835477 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.835457 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99d7dc85-274d-48da-812b-e523fffffc00-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jzclf\" (UID: \"99d7dc85-274d-48da-812b-e523fffffc00\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:17.835533 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.835517 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99d7dc85-274d-48da-812b-e523fffffc00-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jzclf\" (UID: \"99d7dc85-274d-48da-812b-e523fffffc00\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:17.837200 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.837184 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99d7dc85-274d-48da-812b-e523fffffc00-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jzclf\" (UID: \"99d7dc85-274d-48da-812b-e523fffffc00\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:17.837485 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.837466 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99d7dc85-274d-48da-812b-e523fffffc00-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jzclf\" (UID: \"99d7dc85-274d-48da-812b-e523fffffc00\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:17.843263 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.843246 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w67sp\" (UniqueName: \"kubernetes.io/projected/99d7dc85-274d-48da-812b-e523fffffc00-kube-api-access-w67sp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-jzclf\" (UID: \"99d7dc85-274d-48da-812b-e523fffffc00\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:17.968659 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:17.968581 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:18.102334 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:18.102307 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf"] Jul 14 11:29:18.103879 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:29:18.103839 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99d7dc85_274d_48da_812b_e523fffffc00.slice/crio-1eeb82fdda5df231685ea8a30ecd49d18edb2683a62fd6cf5363cb4a879957d9 WatchSource:0}: Error finding container 1eeb82fdda5df231685ea8a30ecd49d18edb2683a62fd6cf5363cb4a879957d9: Status 404 returned error can't find the container with id 1eeb82fdda5df231685ea8a30ecd49d18edb2683a62fd6cf5363cb4a879957d9 Jul 14 11:29:18.461354 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:18.461324 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" event={"ID":"99d7dc85-274d-48da-812b-e523fffffc00","Type":"ContainerStarted","Data":"d8aaa2a0fa0b4208819da67f931226ae113c60a27bc0cb45334a9260a944d2d8"} Jul 14 11:29:18.461496 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:18.461377 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" event={"ID":"99d7dc85-274d-48da-812b-e523fffffc00","Type":"ContainerStarted","Data":"1eeb82fdda5df231685ea8a30ecd49d18edb2683a62fd6cf5363cb4a879957d9"} Jul 14 11:29:18.479930 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:18.479885 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" podStartSLOduration=1.293151253 podStartE2EDuration="1.479869034s" totalImagesPullingTime="186.717781ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:29:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:29:18.478864557 +0000 UTC m=+652.639941018" watchObservedRunningTime="2026-07-14 11:29:18.479869034 +0000 UTC m=+652.640945500" Jul 14 11:29:22.487262 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:22.487233 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:22.487717 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:22.487284 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:22.500127 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:22.500103 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:23.496264 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:23.496237 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-j4tj7" Jul 14 11:29:27.968870 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:27.968840 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:27.969463 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:27.968894 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:27.981275 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:27.981251 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:28.521084 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.521040 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-jzclf" Jul 14 11:29:28.570795 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.570761 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc"] Jul 14 11:29:28.575123 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.575089 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:28.578016 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.577992 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 14 11:29:28.585488 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.585462 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc"] Jul 14 11:29:28.601253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.601232 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:29:28.620576 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.620550 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/841a5533-4023-4b6e-aeec-e8c1aaa74120-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc\" (UID: \"841a5533-4023-4b6e-aeec-e8c1aaa74120\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:28.620686 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.620586 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-255z5\" (UniqueName: \"kubernetes.io/projected/841a5533-4023-4b6e-aeec-e8c1aaa74120-kube-api-access-255z5\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc\" (UID: \"841a5533-4023-4b6e-aeec-e8c1aaa74120\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:28.620686 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.620606 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/841a5533-4023-4b6e-aeec-e8c1aaa74120-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc\" (UID: \"841a5533-4023-4b6e-aeec-e8c1aaa74120\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:28.620802 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.620710 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/841a5533-4023-4b6e-aeec-e8c1aaa74120-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc\" (UID: \"841a5533-4023-4b6e-aeec-e8c1aaa74120\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:28.620802 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.620758 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/841a5533-4023-4b6e-aeec-e8c1aaa74120-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc\" (UID: \"841a5533-4023-4b6e-aeec-e8c1aaa74120\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:28.620802 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.620793 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/841a5533-4023-4b6e-aeec-e8c1aaa74120-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc\" (UID: \"841a5533-4023-4b6e-aeec-e8c1aaa74120\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:28.721981 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.721951 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/841a5533-4023-4b6e-aeec-e8c1aaa74120-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc\" (UID: \"841a5533-4023-4b6e-aeec-e8c1aaa74120\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:28.722076 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.722001 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-255z5\" (UniqueName: \"kubernetes.io/projected/841a5533-4023-4b6e-aeec-e8c1aaa74120-kube-api-access-255z5\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc\" (UID: \"841a5533-4023-4b6e-aeec-e8c1aaa74120\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:28.722150 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.722125 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/841a5533-4023-4b6e-aeec-e8c1aaa74120-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc\" (UID: \"841a5533-4023-4b6e-aeec-e8c1aaa74120\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:28.722204 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.722191 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/841a5533-4023-4b6e-aeec-e8c1aaa74120-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc\" (UID: \"841a5533-4023-4b6e-aeec-e8c1aaa74120\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:28.722258 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.722223 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/841a5533-4023-4b6e-aeec-e8c1aaa74120-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc\" (UID: \"841a5533-4023-4b6e-aeec-e8c1aaa74120\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:28.722315 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.722257 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/841a5533-4023-4b6e-aeec-e8c1aaa74120-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc\" (UID: \"841a5533-4023-4b6e-aeec-e8c1aaa74120\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:28.722563 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.722542 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/841a5533-4023-4b6e-aeec-e8c1aaa74120-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc\" (UID: \"841a5533-4023-4b6e-aeec-e8c1aaa74120\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:28.722751 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.722647 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/841a5533-4023-4b6e-aeec-e8c1aaa74120-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc\" (UID: \"841a5533-4023-4b6e-aeec-e8c1aaa74120\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:28.722942 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.722905 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/841a5533-4023-4b6e-aeec-e8c1aaa74120-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc\" (UID: \"841a5533-4023-4b6e-aeec-e8c1aaa74120\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:28.726147 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.725024 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/841a5533-4023-4b6e-aeec-e8c1aaa74120-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc\" (UID: \"841a5533-4023-4b6e-aeec-e8c1aaa74120\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:28.727852 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.727827 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/841a5533-4023-4b6e-aeec-e8c1aaa74120-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc\" (UID: \"841a5533-4023-4b6e-aeec-e8c1aaa74120\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:28.731139 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.731109 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-255z5\" (UniqueName: \"kubernetes.io/projected/841a5533-4023-4b6e-aeec-e8c1aaa74120-kube-api-access-255z5\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc\" (UID: \"841a5533-4023-4b6e-aeec-e8c1aaa74120\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:28.889472 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:28.889449 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:29.026107 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:29.026078 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc"] Jul 14 11:29:29.027147 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:29:29.027121 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod841a5533_4023_4b6e_aeec_e8c1aaa74120.slice/crio-14f7d7e1b20b0eccf7fcc461f5d438756ba344501429e6cf678eaf642373e70b WatchSource:0}: Error finding container 14f7d7e1b20b0eccf7fcc461f5d438756ba344501429e6cf678eaf642373e70b: Status 404 returned error can't find the container with id 14f7d7e1b20b0eccf7fcc461f5d438756ba344501429e6cf678eaf642373e70b Jul 14 11:29:29.517024 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:29.516943 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" event={"ID":"841a5533-4023-4b6e-aeec-e8c1aaa74120","Type":"ContainerStarted","Data":"fb09b3339bcbfd58cd9c126e7b8682c1cdb85d0b0cd16fa5564046dcaa9f5a21"} Jul 14 11:29:29.517024 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:29.517005 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" event={"ID":"841a5533-4023-4b6e-aeec-e8c1aaa74120","Type":"ContainerStarted","Data":"14f7d7e1b20b0eccf7fcc461f5d438756ba344501429e6cf678eaf642373e70b"} Jul 14 11:29:29.538188 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:29.538143 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" podStartSLOduration=1.3234285940000001 podStartE2EDuration="1.538127213s" totalImagesPullingTime="214.698619ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:29:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:29:29.536696023 +0000 UTC m=+663.697772489" watchObservedRunningTime="2026-07-14 11:29:29.538127213 +0000 UTC m=+663.699203680" Jul 14 11:29:37.904939 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:37.904892 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:29:38.890061 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:38.890020 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:38.890226 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:38.890103 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:38.901962 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:38.901940 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:39.586289 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:39.586263 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc" Jul 14 11:29:45.611785 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:29:45.611756 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:30:26.904761 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:30:26.904680 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:30:27.006344 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:30:27.006295 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:30:28.079263 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:30:28.079221 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:30:28.182437 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:30:28.182402 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:30:51.350709 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:30:51.350676 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:30:51.451222 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:30:51.451183 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:31:04.304006 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:04.303967 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:31:15.458716 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:15.458681 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:31:38.477368 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:38.477328 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:31:38.584289 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:38.584261 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f86cb7fff-npfxr"] Jul 14 11:31:38.584442 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:38.584421 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5f86cb7fff-npfxr" podUID="de2bc2b1-aaaf-4794-84f8-ccf53c646dd3" containerName="manager" containerID="cri-o://20489015c62f237f2fdec56cf59b0c9a260fcda95f8fd61ca6da75bd7388896c" gracePeriod=30 Jul 14 11:31:38.830635 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:38.830611 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f86cb7fff-npfxr" Jul 14 11:31:38.937211 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:38.937183 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de2bc2b1-aaaf-4794-84f8-ccf53c646dd3-webhook-cert\") pod \"de2bc2b1-aaaf-4794-84f8-ccf53c646dd3\" (UID: \"de2bc2b1-aaaf-4794-84f8-ccf53c646dd3\") " Jul 14 11:31:38.937394 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:38.937376 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vl86n\" (UniqueName: \"kubernetes.io/projected/de2bc2b1-aaaf-4794-84f8-ccf53c646dd3-kube-api-access-vl86n\") pod \"de2bc2b1-aaaf-4794-84f8-ccf53c646dd3\" (UID: \"de2bc2b1-aaaf-4794-84f8-ccf53c646dd3\") " Jul 14 11:31:38.939219 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:38.939187 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de2bc2b1-aaaf-4794-84f8-ccf53c646dd3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "de2bc2b1-aaaf-4794-84f8-ccf53c646dd3" (UID: "de2bc2b1-aaaf-4794-84f8-ccf53c646dd3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:31:38.939333 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:38.939265 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de2bc2b1-aaaf-4794-84f8-ccf53c646dd3-kube-api-access-vl86n" (OuterVolumeSpecName: "kube-api-access-vl86n") pod "de2bc2b1-aaaf-4794-84f8-ccf53c646dd3" (UID: "de2bc2b1-aaaf-4794-84f8-ccf53c646dd3"). InnerVolumeSpecName "kube-api-access-vl86n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:31:39.038374 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:39.038316 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vl86n\" (UniqueName: \"kubernetes.io/projected/de2bc2b1-aaaf-4794-84f8-ccf53c646dd3-kube-api-access-vl86n\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:31:39.038374 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:39.038338 2571 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/de2bc2b1-aaaf-4794-84f8-ccf53c646dd3-webhook-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:31:39.176883 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:39.176854 2571 generic.go:356] "Generic (PLEG): container finished" podID="de2bc2b1-aaaf-4794-84f8-ccf53c646dd3" containerID="20489015c62f237f2fdec56cf59b0c9a260fcda95f8fd61ca6da75bd7388896c" exitCode=0 Jul 14 11:31:39.176996 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:39.176895 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f86cb7fff-npfxr" Jul 14 11:31:39.176996 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:39.176937 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f86cb7fff-npfxr" event={"ID":"de2bc2b1-aaaf-4794-84f8-ccf53c646dd3","Type":"ContainerDied","Data":"20489015c62f237f2fdec56cf59b0c9a260fcda95f8fd61ca6da75bd7388896c"} Jul 14 11:31:39.177108 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:39.177016 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f86cb7fff-npfxr" event={"ID":"de2bc2b1-aaaf-4794-84f8-ccf53c646dd3","Type":"ContainerDied","Data":"174f92cdcd6f0947061352058556adcdaf5e30c5be96ec714474f4924687a5c5"} Jul 14 11:31:39.177108 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:39.177040 2571 scope.go:117] "RemoveContainer" containerID="20489015c62f237f2fdec56cf59b0c9a260fcda95f8fd61ca6da75bd7388896c" Jul 14 11:31:39.186816 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:39.186799 2571 scope.go:117] "RemoveContainer" containerID="20489015c62f237f2fdec56cf59b0c9a260fcda95f8fd61ca6da75bd7388896c" Jul 14 11:31:39.187059 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:31:39.187044 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20489015c62f237f2fdec56cf59b0c9a260fcda95f8fd61ca6da75bd7388896c\": container with ID starting with 20489015c62f237f2fdec56cf59b0c9a260fcda95f8fd61ca6da75bd7388896c not found: ID does not exist" containerID="20489015c62f237f2fdec56cf59b0c9a260fcda95f8fd61ca6da75bd7388896c" Jul 14 11:31:39.187099 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:39.187070 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20489015c62f237f2fdec56cf59b0c9a260fcda95f8fd61ca6da75bd7388896c"} err="failed to get container status \"20489015c62f237f2fdec56cf59b0c9a260fcda95f8fd61ca6da75bd7388896c\": rpc error: code = NotFound desc = could not find container \"20489015c62f237f2fdec56cf59b0c9a260fcda95f8fd61ca6da75bd7388896c\": container with ID starting with 20489015c62f237f2fdec56cf59b0c9a260fcda95f8fd61ca6da75bd7388896c not found: ID does not exist" Jul 14 11:31:39.199592 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:39.199567 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f86cb7fff-npfxr"] Jul 14 11:31:39.203355 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:39.203334 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f86cb7fff-npfxr"] Jul 14 11:31:39.907839 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:39.907807 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:31:40.013773 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:40.013745 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f86cb7fff-gcb99"] Jul 14 11:31:40.014171 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:40.014147 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de2bc2b1-aaaf-4794-84f8-ccf53c646dd3" containerName="manager" Jul 14 11:31:40.014171 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:40.014169 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="de2bc2b1-aaaf-4794-84f8-ccf53c646dd3" containerName="manager" Jul 14 11:31:40.014332 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:40.014251 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="de2bc2b1-aaaf-4794-84f8-ccf53c646dd3" containerName="manager" Jul 14 11:31:40.018695 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:40.018679 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f86cb7fff-gcb99" Jul 14 11:31:40.021695 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:40.021678 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 14 11:31:40.023011 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:40.022986 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 14 11:31:40.023148 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:40.023036 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4rv5q\"" Jul 14 11:31:40.026483 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:40.026462 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f86cb7fff-gcb99"] Jul 14 11:31:40.148371 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:40.148343 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzdzm\" (UniqueName: \"kubernetes.io/projected/29adf428-76cd-41cd-9ee6-a84118ecb834-kube-api-access-vzdzm\") pod \"maas-controller-5f86cb7fff-gcb99\" (UID: \"29adf428-76cd-41cd-9ee6-a84118ecb834\") " pod="opendatahub/maas-controller-5f86cb7fff-gcb99" Jul 14 11:31:40.148515 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:40.148378 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/29adf428-76cd-41cd-9ee6-a84118ecb834-webhook-cert\") pod \"maas-controller-5f86cb7fff-gcb99\" (UID: \"29adf428-76cd-41cd-9ee6-a84118ecb834\") " pod="opendatahub/maas-controller-5f86cb7fff-gcb99" Jul 14 11:31:40.249266 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:40.249196 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzdzm\" (UniqueName: \"kubernetes.io/projected/29adf428-76cd-41cd-9ee6-a84118ecb834-kube-api-access-vzdzm\") pod \"maas-controller-5f86cb7fff-gcb99\" (UID: \"29adf428-76cd-41cd-9ee6-a84118ecb834\") " pod="opendatahub/maas-controller-5f86cb7fff-gcb99" Jul 14 11:31:40.249266 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:40.249236 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/29adf428-76cd-41cd-9ee6-a84118ecb834-webhook-cert\") pod \"maas-controller-5f86cb7fff-gcb99\" (UID: \"29adf428-76cd-41cd-9ee6-a84118ecb834\") " pod="opendatahub/maas-controller-5f86cb7fff-gcb99" Jul 14 11:31:40.251693 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:40.251663 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/29adf428-76cd-41cd-9ee6-a84118ecb834-webhook-cert\") pod \"maas-controller-5f86cb7fff-gcb99\" (UID: \"29adf428-76cd-41cd-9ee6-a84118ecb834\") " pod="opendatahub/maas-controller-5f86cb7fff-gcb99" Jul 14 11:31:40.258527 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:40.258499 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzdzm\" (UniqueName: \"kubernetes.io/projected/29adf428-76cd-41cd-9ee6-a84118ecb834-kube-api-access-vzdzm\") pod \"maas-controller-5f86cb7fff-gcb99\" (UID: \"29adf428-76cd-41cd-9ee6-a84118ecb834\") " pod="opendatahub/maas-controller-5f86cb7fff-gcb99" Jul 14 11:31:40.329747 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:40.329725 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f86cb7fff-gcb99" Jul 14 11:31:40.403831 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:40.403801 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de2bc2b1-aaaf-4794-84f8-ccf53c646dd3" path="/var/lib/kubelet/pods/de2bc2b1-aaaf-4794-84f8-ccf53c646dd3/volumes" Jul 14 11:31:40.458009 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:40.457983 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f86cb7fff-gcb99"] Jul 14 11:31:40.459328 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:31:40.459300 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29adf428_76cd_41cd_9ee6_a84118ecb834.slice/crio-b7efa218d3418d0316b457214e4319234ffa2c6e52710edc0b2de0d71a72709d WatchSource:0}: Error finding container b7efa218d3418d0316b457214e4319234ffa2c6e52710edc0b2de0d71a72709d: Status 404 returned error can't find the container with id b7efa218d3418d0316b457214e4319234ffa2c6e52710edc0b2de0d71a72709d Jul 14 11:31:41.189617 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:41.189535 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f86cb7fff-gcb99" event={"ID":"29adf428-76cd-41cd-9ee6-a84118ecb834","Type":"ContainerStarted","Data":"67f9167a928b2d549f13d3766c7e2e202bcde80be4d0813833f6c3bcb300cb3a"} Jul 14 11:31:41.189617 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:41.189587 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f86cb7fff-gcb99" event={"ID":"29adf428-76cd-41cd-9ee6-a84118ecb834","Type":"ContainerStarted","Data":"b7efa218d3418d0316b457214e4319234ffa2c6e52710edc0b2de0d71a72709d"} Jul 14 11:31:41.190023 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:41.189686 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f86cb7fff-gcb99" Jul 14 11:31:41.209590 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:41.209543 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f86cb7fff-gcb99" podStartSLOduration=1.8217422829999999 podStartE2EDuration="2.209531023s" totalImagesPullingTime="387.78874ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:31:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:31:41.207711987 +0000 UTC m=+795.368788452" watchObservedRunningTime="2026-07-14 11:31:41.209531023 +0000 UTC m=+795.370607490" Jul 14 11:31:52.197487 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:31:52.197407 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f86cb7fff-gcb99" Jul 14 11:32:15.685157 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:32:15.685120 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:32:15.840606 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:32:15.840569 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:32:15.964106 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:32:15.964026 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:32:22.098013 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:32:22.097977 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:32:22.243768 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:32:22.243732 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:32:22.375915 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:32:22.375833 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:32:22.484906 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:32:22.484872 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:32:29.235901 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:32:29.235863 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:32:29.340073 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:32:29.340034 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:33:26.339867 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:33:26.339836 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 11:33:26.341070 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:33:26.341034 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 11:33:57.791713 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:33:57.791676 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:34:27.363226 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:34:27.363187 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:35:27.119894 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:35:27.119860 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:35:58.079396 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:35:58.079362 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:35:58.180437 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:35:58.180408 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:37:01.896793 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:37:01.896758 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:38:26.373271 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:38:26.373236 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 11:38:26.375578 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:38:26.375557 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 11:39:32.259412 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:39:32.259337 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:41:00.085250 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:41:00.085176 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:41:00.184213 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:41:00.184179 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:42:16.354823 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:42:16.354784 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:42:16.451859 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:42:16.451831 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:43:19.747582 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:19.747547 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:43:19.844905 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:19.844875 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:43:25.897442 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:25.897412 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:43:25.962926 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:25.962890 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:43:26.077500 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.077460 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk"] Jul 14 11:43:26.081361 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.081341 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.084845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.084826 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e8442e03-openshift-default-dockercfg-57dzq\"" Jul 14 11:43:26.100813 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.100792 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk"] Jul 14 11:43:26.118502 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.118477 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:43:26.208750 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.208663 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-credential-socket\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.208750 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.208707 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87a04dc7-74af-483e-b3b2-69146edbd593-istiod-ca-cert\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.208750 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.208730 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-workload-socket\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.208967 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.208757 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-istio-data\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.208967 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.208790 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-workload-certs\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.208967 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.208806 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87a04dc7-74af-483e-b3b2-69146edbd593-istio-podinfo\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.208967 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.208852 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87a04dc7-74af-483e-b3b2-69146edbd593-istio-token\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.208967 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.208894 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-istio-envoy\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.208967 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.208960 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgqrx\" (UniqueName: \"kubernetes.io/projected/87a04dc7-74af-483e-b3b2-69146edbd593-kube-api-access-kgqrx\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.216682 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.216651 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:43:26.309473 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.309438 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87a04dc7-74af-483e-b3b2-69146edbd593-istio-token\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.309473 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.309473 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-istio-envoy\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.309716 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.309610 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgqrx\" (UniqueName: \"kubernetes.io/projected/87a04dc7-74af-483e-b3b2-69146edbd593-kube-api-access-kgqrx\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.309768 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.309738 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-credential-socket\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.309817 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.309784 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87a04dc7-74af-483e-b3b2-69146edbd593-istiod-ca-cert\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.309857 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.309815 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-workload-socket\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.309912 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.309853 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-istio-data\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.309912 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.309887 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-workload-certs\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.310016 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.309915 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87a04dc7-74af-483e-b3b2-69146edbd593-istio-podinfo\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.310158 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.310133 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-credential-socket\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.310243 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.310217 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-istio-data\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.310343 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.310319 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-workload-socket\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.310399 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.310351 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-workload-certs\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.310456 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.310436 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87a04dc7-74af-483e-b3b2-69146edbd593-istiod-ca-cert\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.311755 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.311732 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-istio-envoy\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.312253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.312233 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87a04dc7-74af-483e-b3b2-69146edbd593-istio-podinfo\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.318771 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.318747 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgqrx\" (UniqueName: \"kubernetes.io/projected/87a04dc7-74af-483e-b3b2-69146edbd593-kube-api-access-kgqrx\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.318771 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.318769 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87a04dc7-74af-483e-b3b2-69146edbd593-istio-token\") pod \"e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.398114 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.398074 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e8442e03-openshift-default-dockercfg-57dzq\"" Jul 14 11:43:26.402845 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.402821 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:26.413182 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.413150 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 11:43:26.417822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.417791 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 11:43:26.556361 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.556333 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk"] Jul 14 11:43:26.558143 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:43:26.558117 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87a04dc7_74af_483e_b3b2_69146edbd593.slice/crio-026f073acc90fc7dc8e7c97e2eee7e7383ad9c453f315640822236bba1931465 WatchSource:0}: Error finding container 026f073acc90fc7dc8e7c97e2eee7e7383ad9c453f315640822236bba1931465: Status 404 returned error can't find the container with id 026f073acc90fc7dc8e7c97e2eee7e7383ad9c453f315640822236bba1931465 Jul 14 11:43:26.560726 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.560684 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:43:26.560829 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.560773 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:43:26.560829 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.560815 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:43:26.877426 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.877390 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" event={"ID":"87a04dc7-74af-483e-b3b2-69146edbd593","Type":"ContainerStarted","Data":"cee5f6fe62421afb881b44ef1f169e0c5ecbe8103c29afbdaf83f9412289f3cd"} Jul 14 11:43:26.877651 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.877461 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" event={"ID":"87a04dc7-74af-483e-b3b2-69146edbd593","Type":"ContainerStarted","Data":"026f073acc90fc7dc8e7c97e2eee7e7383ad9c453f315640822236bba1931465"} Jul 14 11:43:26.900762 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:26.900709 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" podStartSLOduration=0.900692502 podStartE2EDuration="900.692502ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:43:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:43:26.898712973 +0000 UTC m=+1501.059789443" watchObservedRunningTime="2026-07-14 11:43:26.900692502 +0000 UTC m=+1501.061768970" Jul 14 11:43:27.244841 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.244749 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:43:27.394144 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.394103 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x"] Jul 14 11:43:27.397821 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.397803 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x" Jul 14 11:43:27.403398 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.403374 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:27.403822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.403804 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e8442e03-serving-cert\"" Jul 14 11:43:27.408539 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.408520 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:27.419659 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.419622 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x"] Jul 14 11:43:27.466869 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.466838 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:43:27.522266 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.522202 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad371160-3050-4cbd-b27e-81884641bbb0-maas-api-tls\") pod \"maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x\" (UID: \"ad371160-3050-4cbd-b27e-81884641bbb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x" Jul 14 11:43:27.522266 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.522241 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5p2v\" (UniqueName: \"kubernetes.io/projected/ad371160-3050-4cbd-b27e-81884641bbb0-kube-api-access-l5p2v\") pod \"maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x\" (UID: \"ad371160-3050-4cbd-b27e-81884641bbb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x" Jul 14 11:43:27.522401 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.522374 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad371160-3050-4cbd-b27e-81884641bbb0-openshift-service-ca\") pod \"maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x\" (UID: \"ad371160-3050-4cbd-b27e-81884641bbb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x" Jul 14 11:43:27.560317 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.560283 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:43:27.623364 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.623332 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5p2v\" (UniqueName: \"kubernetes.io/projected/ad371160-3050-4cbd-b27e-81884641bbb0-kube-api-access-l5p2v\") pod \"maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x\" (UID: \"ad371160-3050-4cbd-b27e-81884641bbb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x" Jul 14 11:43:27.623485 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.623439 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad371160-3050-4cbd-b27e-81884641bbb0-openshift-service-ca\") pod \"maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x\" (UID: \"ad371160-3050-4cbd-b27e-81884641bbb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x" Jul 14 11:43:27.623531 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.623507 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad371160-3050-4cbd-b27e-81884641bbb0-maas-api-tls\") pod \"maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x\" (UID: \"ad371160-3050-4cbd-b27e-81884641bbb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x" Jul 14 11:43:27.624142 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.624115 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad371160-3050-4cbd-b27e-81884641bbb0-openshift-service-ca\") pod \"maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x\" (UID: \"ad371160-3050-4cbd-b27e-81884641bbb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x" Jul 14 11:43:27.625836 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.625816 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad371160-3050-4cbd-b27e-81884641bbb0-maas-api-tls\") pod \"maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x\" (UID: \"ad371160-3050-4cbd-b27e-81884641bbb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x" Jul 14 11:43:27.633300 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.633277 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5p2v\" (UniqueName: \"kubernetes.io/projected/ad371160-3050-4cbd-b27e-81884641bbb0-kube-api-access-l5p2v\") pod \"maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x\" (UID: \"ad371160-3050-4cbd-b27e-81884641bbb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x" Jul 14 11:43:27.708911 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.708886 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x" Jul 14 11:43:27.857380 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.857354 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x"] Jul 14 11:43:27.858931 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:43:27.858908 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad371160_3050_4cbd_b27e_81884641bbb0.slice/crio-933120024ab6e1ab7e940afac42ca30e2d2834c7afc25e3274ad0c645e9a07d8 WatchSource:0}: Error finding container 933120024ab6e1ab7e940afac42ca30e2d2834c7afc25e3274ad0c645e9a07d8: Status 404 returned error can't find the container with id 933120024ab6e1ab7e940afac42ca30e2d2834c7afc25e3274ad0c645e9a07d8 Jul 14 11:43:27.860372 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.860358 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 11:43:27.887097 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.887066 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x" event={"ID":"ad371160-3050-4cbd-b27e-81884641bbb0","Type":"ContainerStarted","Data":"933120024ab6e1ab7e940afac42ca30e2d2834c7afc25e3274ad0c645e9a07d8"} Jul 14 11:43:27.887380 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.887360 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:27.888306 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:27.888287 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:28.279365 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:28.279330 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:43:28.388799 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:28.388766 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:43:28.894392 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:28.894353 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x" event={"ID":"ad371160-3050-4cbd-b27e-81884641bbb0","Type":"ContainerStarted","Data":"d346bf4fe192c6889b6b0a03a6981d494f148170ac037f5f2c24f6ce1130fc70"} Jul 14 11:43:28.894652 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:28.894612 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x" Jul 14 11:43:28.941930 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:28.941882 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x" podStartSLOduration=1.373530208 podStartE2EDuration="1.941868351s" totalImagesPullingTime="568.338143ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:43:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:43:28.938085473 +0000 UTC m=+1503.099161939" watchObservedRunningTime="2026-07-14 11:43:28.941868351 +0000 UTC m=+1503.102944818" Jul 14 11:43:29.683370 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:29.683339 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:43:34.905032 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:34.905005 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x" Jul 14 11:43:42.888354 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:42.888326 2571 patch_prober.go:28] interesting pod/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 14 11:43:42.888710 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:42.888384 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" podUID="87a04dc7-74af-483e-b3b2-69146edbd593" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 14 11:43:42.971273 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:42.971243 2571 generic.go:356] "Generic (PLEG): container finished" podID="87a04dc7-74af-483e-b3b2-69146edbd593" containerID="cee5f6fe62421afb881b44ef1f169e0c5ecbe8103c29afbdaf83f9412289f3cd" exitCode=0 Jul 14 11:43:42.971408 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:42.971287 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" event={"ID":"87a04dc7-74af-483e-b3b2-69146edbd593","Type":"ContainerDied","Data":"cee5f6fe62421afb881b44ef1f169e0c5ecbe8103c29afbdaf83f9412289f3cd"} Jul 14 11:43:42.971683 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:42.971665 2571 scope.go:117] "RemoveContainer" containerID="cee5f6fe62421afb881b44ef1f169e0c5ecbe8103c29afbdaf83f9412289f3cd" Jul 14 11:43:42.972727 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:42.972694 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:43:42.972803 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:42.972785 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:43:42.972855 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:42.972839 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:43:43.982494 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:43.982452 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" event={"ID":"87a04dc7-74af-483e-b3b2-69146edbd593","Type":"ContainerStarted","Data":"a4f6729cf47adfbddb50a61f3b391d34dd030b0c6eb18717339e48482def52e3"} Jul 14 11:43:44.403738 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:44.403705 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:44.403917 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:44.403819 2571 patch_prober.go:28] interesting pod/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 14 11:43:44.403917 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:44.403879 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" podUID="87a04dc7-74af-483e-b3b2-69146edbd593" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 14 11:43:45.404245 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:45.404208 2571 patch_prober.go:28] interesting pod/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 14 11:43:45.404620 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:45.404275 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" podUID="87a04dc7-74af-483e-b3b2-69146edbd593" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 14 11:43:46.403501 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:46.403466 2571 patch_prober.go:28] interesting pod/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 14 11:43:46.403722 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:46.403537 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" podUID="87a04dc7-74af-483e-b3b2-69146edbd593" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 14 11:43:47.403829 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:47.403786 2571 patch_prober.go:28] interesting pod/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 14 11:43:47.404280 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:47.403848 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" podUID="87a04dc7-74af-483e-b3b2-69146edbd593" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 14 11:43:48.406978 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:48.406952 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:48.407373 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:48.407225 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:43:48.407912 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:43:48.407896 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:44:18.432651 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:18.432603 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:44:43.019917 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.019883 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x"] Jul 14 11:44:43.020266 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.020088 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x" podUID="ad371160-3050-4cbd-b27e-81884641bbb0" containerName="maas-api" containerID="cri-o://d346bf4fe192c6889b6b0a03a6981d494f148170ac037f5f2c24f6ce1130fc70" gracePeriod=30 Jul 14 11:44:43.050865 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.050835 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:44:43.269522 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.269498 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x" Jul 14 11:44:43.303582 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.303516 2571 generic.go:356] "Generic (PLEG): container finished" podID="ad371160-3050-4cbd-b27e-81884641bbb0" containerID="d346bf4fe192c6889b6b0a03a6981d494f148170ac037f5f2c24f6ce1130fc70" exitCode=0 Jul 14 11:44:43.303582 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.303553 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x" event={"ID":"ad371160-3050-4cbd-b27e-81884641bbb0","Type":"ContainerDied","Data":"d346bf4fe192c6889b6b0a03a6981d494f148170ac037f5f2c24f6ce1130fc70"} Jul 14 11:44:43.303797 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.303614 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x" event={"ID":"ad371160-3050-4cbd-b27e-81884641bbb0","Type":"ContainerDied","Data":"933120024ab6e1ab7e940afac42ca30e2d2834c7afc25e3274ad0c645e9a07d8"} Jul 14 11:44:43.303797 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.303563 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x" Jul 14 11:44:43.303797 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.303656 2571 scope.go:117] "RemoveContainer" containerID="d346bf4fe192c6889b6b0a03a6981d494f148170ac037f5f2c24f6ce1130fc70" Jul 14 11:44:43.312657 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.312619 2571 scope.go:117] "RemoveContainer" containerID="d346bf4fe192c6889b6b0a03a6981d494f148170ac037f5f2c24f6ce1130fc70" Jul 14 11:44:43.312923 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:44:43.312897 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d346bf4fe192c6889b6b0a03a6981d494f148170ac037f5f2c24f6ce1130fc70\": container with ID starting with d346bf4fe192c6889b6b0a03a6981d494f148170ac037f5f2c24f6ce1130fc70 not found: ID does not exist" containerID="d346bf4fe192c6889b6b0a03a6981d494f148170ac037f5f2c24f6ce1130fc70" Jul 14 11:44:43.313007 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.312934 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d346bf4fe192c6889b6b0a03a6981d494f148170ac037f5f2c24f6ce1130fc70"} err="failed to get container status \"d346bf4fe192c6889b6b0a03a6981d494f148170ac037f5f2c24f6ce1130fc70\": rpc error: code = NotFound desc = could not find container \"d346bf4fe192c6889b6b0a03a6981d494f148170ac037f5f2c24f6ce1130fc70\": container with ID starting with d346bf4fe192c6889b6b0a03a6981d494f148170ac037f5f2c24f6ce1130fc70 not found: ID does not exist" Jul 14 11:44:43.378883 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.378860 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad371160-3050-4cbd-b27e-81884641bbb0-maas-api-tls\") pod \"ad371160-3050-4cbd-b27e-81884641bbb0\" (UID: \"ad371160-3050-4cbd-b27e-81884641bbb0\") " Jul 14 11:44:43.379017 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.379002 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5p2v\" (UniqueName: \"kubernetes.io/projected/ad371160-3050-4cbd-b27e-81884641bbb0-kube-api-access-l5p2v\") pod \"ad371160-3050-4cbd-b27e-81884641bbb0\" (UID: \"ad371160-3050-4cbd-b27e-81884641bbb0\") " Jul 14 11:44:43.379058 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.379050 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad371160-3050-4cbd-b27e-81884641bbb0-openshift-service-ca\") pod \"ad371160-3050-4cbd-b27e-81884641bbb0\" (UID: \"ad371160-3050-4cbd-b27e-81884641bbb0\") " Jul 14 11:44:43.379418 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.379392 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad371160-3050-4cbd-b27e-81884641bbb0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ad371160-3050-4cbd-b27e-81884641bbb0" (UID: "ad371160-3050-4cbd-b27e-81884641bbb0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:44:43.380866 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.380842 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad371160-3050-4cbd-b27e-81884641bbb0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ad371160-3050-4cbd-b27e-81884641bbb0" (UID: "ad371160-3050-4cbd-b27e-81884641bbb0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 11:44:43.380980 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.380962 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad371160-3050-4cbd-b27e-81884641bbb0-kube-api-access-l5p2v" (OuterVolumeSpecName: "kube-api-access-l5p2v") pod "ad371160-3050-4cbd-b27e-81884641bbb0" (UID: "ad371160-3050-4cbd-b27e-81884641bbb0"). InnerVolumeSpecName "kube-api-access-l5p2v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:44:43.480478 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.480424 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5p2v\" (UniqueName: \"kubernetes.io/projected/ad371160-3050-4cbd-b27e-81884641bbb0-kube-api-access-l5p2v\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:44:43.480478 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.480450 2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad371160-3050-4cbd-b27e-81884641bbb0-openshift-service-ca\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:44:43.480478 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.480463 2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad371160-3050-4cbd-b27e-81884641bbb0-maas-api-tls\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:44:43.627554 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.627527 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x"] Jul 14 11:44:43.631537 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:43.631516 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8442e03-6dfbbbd45d-kcv4x"] Jul 14 11:44:44.403692 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:44.403659 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad371160-3050-4cbd-b27e-81884641bbb0" path="/var/lib/kubelet/pods/ad371160-3050-4cbd-b27e-81884641bbb0/volumes" Jul 14 11:44:47.522479 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:47.522446 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:44:53.334120 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:53.334088 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:44:53.434178 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:53.434136 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:44:57.736441 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:57.736406 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk"] Jul 14 11:44:57.736848 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:57.736601 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" podUID="87a04dc7-74af-483e-b3b2-69146edbd593" containerName="istio-proxy" containerID="cri-o://a4f6729cf47adfbddb50a61f3b391d34dd030b0c6eb18717339e48482def52e3" gracePeriod=30 Jul 14 11:44:57.780392 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:57.780365 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:44:58.116091 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.116060 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:44:58.245009 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.244979 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:44:58.353164 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.353118 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl"] Jul 14 11:44:58.353772 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.353753 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad371160-3050-4cbd-b27e-81884641bbb0" containerName="maas-api" Jul 14 11:44:58.353822 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.353776 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad371160-3050-4cbd-b27e-81884641bbb0" containerName="maas-api" Jul 14 11:44:58.353904 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.353892 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad371160-3050-4cbd-b27e-81884641bbb0" containerName="maas-api" Jul 14 11:44:58.357243 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.357225 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.364774 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.364755 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-f22ceb3a-gw-openshift-default-dockercfg-5lldm\"" Jul 14 11:44:58.380795 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.380738 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl"] Jul 14 11:44:58.505558 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.505513 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-workload-certs\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.505783 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.505567 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-token\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.505783 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.505738 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-workload-socket\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.505998 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.505973 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-credential-socket\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.506068 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.506033 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6vc4\" (UniqueName: \"kubernetes.io/projected/3e6b8635-e3ea-4b43-96a1-60970009617c-kube-api-access-x6vc4\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.506129 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.506077 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e6b8635-e3ea-4b43-96a1-60970009617c-istiod-ca-cert\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.506183 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.506139 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-podinfo\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.506183 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.506174 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-envoy\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.506289 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.506207 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-data\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.517337 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.517275 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:44:58.607443 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.607413 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-token\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.607603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.607457 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-workload-socket\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.607603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.607509 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-credential-socket\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.607603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.607544 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6vc4\" (UniqueName: \"kubernetes.io/projected/3e6b8635-e3ea-4b43-96a1-60970009617c-kube-api-access-x6vc4\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.607603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.607581 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e6b8635-e3ea-4b43-96a1-60970009617c-istiod-ca-cert\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.607901 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.607650 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-podinfo\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.607901 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.607684 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-envoy\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.607901 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.607725 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-data\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.607901 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.607783 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-workload-certs\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.608216 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.608184 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-credential-socket\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.608339 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.608216 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-workload-socket\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.608339 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.608273 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-workload-certs\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.608339 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.608304 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-data\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.608573 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.608553 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e6b8635-e3ea-4b43-96a1-60970009617c-istiod-ca-cert\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.610078 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.610056 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-envoy\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.610175 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.610129 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-podinfo\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.616449 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.616422 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-token\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.616624 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.616606 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6vc4\" (UniqueName: \"kubernetes.io/projected/3e6b8635-e3ea-4b43-96a1-60970009617c-kube-api-access-x6vc4\") pod \"e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.667825 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.667773 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:58.800606 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.800578 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl"] Jul 14 11:44:58.801020 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:44:58.800996 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e6b8635_e3ea_4b43_96a1_60970009617c.slice/crio-d16216966248808e0f97c0c5ba2ac2dd4d8cd9381f6c056863401f9b64331857 WatchSource:0}: Error finding container d16216966248808e0f97c0c5ba2ac2dd4d8cd9381f6c056863401f9b64331857: Status 404 returned error can't find the container with id d16216966248808e0f97c0c5ba2ac2dd4d8cd9381f6c056863401f9b64331857 Jul 14 11:44:58.803225 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.803182 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:44:58.803290 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.803277 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:44:58.803341 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:58.803325 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:44:59.393116 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:59.393069 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" event={"ID":"3e6b8635-e3ea-4b43-96a1-60970009617c","Type":"ContainerStarted","Data":"9cda09da178f7a6f351091f6d912f9e486c2ee5af34123a4bb9c6f73e07fc0dd"} Jul 14 11:44:59.393309 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:59.393152 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" event={"ID":"3e6b8635-e3ea-4b43-96a1-60970009617c","Type":"ContainerStarted","Data":"d16216966248808e0f97c0c5ba2ac2dd4d8cd9381f6c056863401f9b64331857"} Jul 14 11:44:59.420496 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:59.420452 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" podStartSLOduration=1.420437946 podStartE2EDuration="1.420437946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:44:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:44:59.417975756 +0000 UTC m=+1593.579052265" watchObservedRunningTime="2026-07-14 11:44:59.420437946 +0000 UTC m=+1593.581514411" Jul 14 11:44:59.666698 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:59.666610 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:44:59.668607 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:59.668586 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:59.672944 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:59.672920 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:44:59.796453 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:44:59.796425 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:45:00.298177 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:00.298143 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:45:00.397710 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:00.397681 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:45:00.402901 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:00.402875 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:45:00.403042 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:00.402948 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:45:00.501410 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:00.501381 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:45:02.991998 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:02.991972 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:45:03.152450 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.152413 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-credential-socket\") pod \"87a04dc7-74af-483e-b3b2-69146edbd593\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " Jul 14 11:45:03.152668 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.152470 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87a04dc7-74af-483e-b3b2-69146edbd593-istiod-ca-cert\") pod \"87a04dc7-74af-483e-b3b2-69146edbd593\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " Jul 14 11:45:03.152668 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.152537 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87a04dc7-74af-483e-b3b2-69146edbd593-istio-token\") pod \"87a04dc7-74af-483e-b3b2-69146edbd593\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " Jul 14 11:45:03.152668 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.152583 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87a04dc7-74af-483e-b3b2-69146edbd593-istio-podinfo\") pod \"87a04dc7-74af-483e-b3b2-69146edbd593\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " Jul 14 11:45:03.152668 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.152614 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-workload-certs\") pod \"87a04dc7-74af-483e-b3b2-69146edbd593\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " Jul 14 11:45:03.152908 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.152670 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgqrx\" (UniqueName: \"kubernetes.io/projected/87a04dc7-74af-483e-b3b2-69146edbd593-kube-api-access-kgqrx\") pod \"87a04dc7-74af-483e-b3b2-69146edbd593\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " Jul 14 11:45:03.152908 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.152729 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-istio-envoy\") pod \"87a04dc7-74af-483e-b3b2-69146edbd593\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " Jul 14 11:45:03.152908 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.152754 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "87a04dc7-74af-483e-b3b2-69146edbd593" (UID: "87a04dc7-74af-483e-b3b2-69146edbd593"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:45:03.152908 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.152787 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-istio-data\") pod \"87a04dc7-74af-483e-b3b2-69146edbd593\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " Jul 14 11:45:03.152908 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.152827 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-workload-socket\") pod \"87a04dc7-74af-483e-b3b2-69146edbd593\" (UID: \"87a04dc7-74af-483e-b3b2-69146edbd593\") " Jul 14 11:45:03.153181 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.152905 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87a04dc7-74af-483e-b3b2-69146edbd593-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "87a04dc7-74af-483e-b3b2-69146edbd593" (UID: "87a04dc7-74af-483e-b3b2-69146edbd593"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:45:03.153181 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.152922 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "87a04dc7-74af-483e-b3b2-69146edbd593" (UID: "87a04dc7-74af-483e-b3b2-69146edbd593"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:45:03.157460 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.156436 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-workload-certs\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:45:03.157460 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.156474 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-credential-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:45:03.157460 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.156503 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87a04dc7-74af-483e-b3b2-69146edbd593-istiod-ca-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:45:03.157460 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.157252 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87a04dc7-74af-483e-b3b2-69146edbd593-istio-token" (OuterVolumeSpecName: "istio-token") pod "87a04dc7-74af-483e-b3b2-69146edbd593" (UID: "87a04dc7-74af-483e-b3b2-69146edbd593"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:45:03.157778 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.157515 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "87a04dc7-74af-483e-b3b2-69146edbd593" (UID: "87a04dc7-74af-483e-b3b2-69146edbd593"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:45:03.157778 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.157725 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-istio-data" (OuterVolumeSpecName: "istio-data") pod "87a04dc7-74af-483e-b3b2-69146edbd593" (UID: "87a04dc7-74af-483e-b3b2-69146edbd593"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:45:03.161738 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.159789 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "87a04dc7-74af-483e-b3b2-69146edbd593" (UID: "87a04dc7-74af-483e-b3b2-69146edbd593"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:45:03.166814 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.164851 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87a04dc7-74af-483e-b3b2-69146edbd593-kube-api-access-kgqrx" (OuterVolumeSpecName: "kube-api-access-kgqrx") pod "87a04dc7-74af-483e-b3b2-69146edbd593" (UID: "87a04dc7-74af-483e-b3b2-69146edbd593"). InnerVolumeSpecName "kube-api-access-kgqrx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:45:03.167287 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.167261 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/87a04dc7-74af-483e-b3b2-69146edbd593-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "87a04dc7-74af-483e-b3b2-69146edbd593" (UID: "87a04dc7-74af-483e-b3b2-69146edbd593"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 11:45:03.257038 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.257003 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87a04dc7-74af-483e-b3b2-69146edbd593-istio-token\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:45:03.257038 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.257033 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87a04dc7-74af-483e-b3b2-69146edbd593-istio-podinfo\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:45:03.257038 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.257042 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgqrx\" (UniqueName: \"kubernetes.io/projected/87a04dc7-74af-483e-b3b2-69146edbd593-kube-api-access-kgqrx\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:45:03.257304 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.257051 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-istio-envoy\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:45:03.257304 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.257061 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-istio-data\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:45:03.257304 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.257068 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87a04dc7-74af-483e-b3b2-69146edbd593-workload-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:45:03.414528 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.414495 2571 generic.go:356] "Generic (PLEG): container finished" podID="87a04dc7-74af-483e-b3b2-69146edbd593" containerID="a4f6729cf47adfbddb50a61f3b391d34dd030b0c6eb18717339e48482def52e3" exitCode=0 Jul 14 11:45:03.414717 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.414559 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" Jul 14 11:45:03.414717 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.414582 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" event={"ID":"87a04dc7-74af-483e-b3b2-69146edbd593","Type":"ContainerDied","Data":"a4f6729cf47adfbddb50a61f3b391d34dd030b0c6eb18717339e48482def52e3"} Jul 14 11:45:03.414717 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.414663 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk" event={"ID":"87a04dc7-74af-483e-b3b2-69146edbd593","Type":"ContainerDied","Data":"026f073acc90fc7dc8e7c97e2eee7e7383ad9c453f315640822236bba1931465"} Jul 14 11:45:03.414717 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.414692 2571 scope.go:117] "RemoveContainer" containerID="a4f6729cf47adfbddb50a61f3b391d34dd030b0c6eb18717339e48482def52e3" Jul 14 11:45:03.424000 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.423983 2571 scope.go:117] "RemoveContainer" containerID="cee5f6fe62421afb881b44ef1f169e0c5ecbe8103c29afbdaf83f9412289f3cd" Jul 14 11:45:03.432051 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.432034 2571 scope.go:117] "RemoveContainer" containerID="a4f6729cf47adfbddb50a61f3b391d34dd030b0c6eb18717339e48482def52e3" Jul 14 11:45:03.432293 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:45:03.432272 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4f6729cf47adfbddb50a61f3b391d34dd030b0c6eb18717339e48482def52e3\": container with ID starting with a4f6729cf47adfbddb50a61f3b391d34dd030b0c6eb18717339e48482def52e3 not found: ID does not exist" containerID="a4f6729cf47adfbddb50a61f3b391d34dd030b0c6eb18717339e48482def52e3" Jul 14 11:45:03.432334 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.432302 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4f6729cf47adfbddb50a61f3b391d34dd030b0c6eb18717339e48482def52e3"} err="failed to get container status \"a4f6729cf47adfbddb50a61f3b391d34dd030b0c6eb18717339e48482def52e3\": rpc error: code = NotFound desc = could not find container \"a4f6729cf47adfbddb50a61f3b391d34dd030b0c6eb18717339e48482def52e3\": container with ID starting with a4f6729cf47adfbddb50a61f3b391d34dd030b0c6eb18717339e48482def52e3 not found: ID does not exist" Jul 14 11:45:03.432334 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.432318 2571 scope.go:117] "RemoveContainer" containerID="cee5f6fe62421afb881b44ef1f169e0c5ecbe8103c29afbdaf83f9412289f3cd" Jul 14 11:45:03.432541 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:45:03.432524 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cee5f6fe62421afb881b44ef1f169e0c5ecbe8103c29afbdaf83f9412289f3cd\": container with ID starting with cee5f6fe62421afb881b44ef1f169e0c5ecbe8103c29afbdaf83f9412289f3cd not found: ID does not exist" containerID="cee5f6fe62421afb881b44ef1f169e0c5ecbe8103c29afbdaf83f9412289f3cd" Jul 14 11:45:03.432580 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.432548 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cee5f6fe62421afb881b44ef1f169e0c5ecbe8103c29afbdaf83f9412289f3cd"} err="failed to get container status \"cee5f6fe62421afb881b44ef1f169e0c5ecbe8103c29afbdaf83f9412289f3cd\": rpc error: code = NotFound desc = could not find container \"cee5f6fe62421afb881b44ef1f169e0c5ecbe8103c29afbdaf83f9412289f3cd\": container with ID starting with cee5f6fe62421afb881b44ef1f169e0c5ecbe8103c29afbdaf83f9412289f3cd not found: ID does not exist" Jul 14 11:45:03.438896 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.438876 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk"] Jul 14 11:45:03.443075 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:03.443056 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e8442e03-openshift-default-7486c8f78f-l76zk"] Jul 14 11:45:04.404119 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:04.404086 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87a04dc7-74af-483e-b3b2-69146edbd593" path="/var/lib/kubelet/pods/87a04dc7-74af-483e-b3b2-69146edbd593/volumes" Jul 14 11:45:09.651430 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:09.651398 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:45:09.710588 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:09.710562 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:45:10.455638 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:10.455597 2571 generic.go:356] "Generic (PLEG): container finished" podID="3e6b8635-e3ea-4b43-96a1-60970009617c" containerID="9cda09da178f7a6f351091f6d912f9e486c2ee5af34123a4bb9c6f73e07fc0dd" exitCode=0 Jul 14 11:45:10.455812 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:10.455673 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" event={"ID":"3e6b8635-e3ea-4b43-96a1-60970009617c","Type":"ContainerDied","Data":"9cda09da178f7a6f351091f6d912f9e486c2ee5af34123a4bb9c6f73e07fc0dd"} Jul 14 11:45:10.456062 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:10.456045 2571 scope.go:117] "RemoveContainer" containerID="9cda09da178f7a6f351091f6d912f9e486c2ee5af34123a4bb9c6f73e07fc0dd" Jul 14 11:45:10.457036 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:10.457009 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:45:10.457117 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:10.457090 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:45:10.457196 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:10.457180 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:45:11.463202 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:11.463171 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" event={"ID":"3e6b8635-e3ea-4b43-96a1-60970009617c","Type":"ContainerStarted","Data":"e33624dcd6981b1c37a676a3ec8389d754964f3bd2524da2486d77c00cb4f861"} Jul 14 11:45:11.669121 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:11.669086 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:45:11.669522 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:11.669492 2571 patch_prober.go:28] interesting pod/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 14 11:45:11.669752 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:11.669725 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" podUID="3e6b8635-e3ea-4b43-96a1-60970009617c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 14 11:45:12.668736 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:12.668700 2571 patch_prober.go:28] interesting pod/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 14 11:45:12.669223 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:12.668764 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" podUID="3e6b8635-e3ea-4b43-96a1-60970009617c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 14 11:45:13.483484 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:13.483444 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:45:13.668515 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:13.668106 2571 patch_prober.go:28] interesting pod/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 14 11:45:13.668515 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:13.668164 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" podUID="3e6b8635-e3ea-4b43-96a1-60970009617c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 14 11:45:14.669017 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:14.668980 2571 patch_prober.go:28] interesting pod/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 14 11:45:14.669609 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:14.669578 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" podUID="3e6b8635-e3ea-4b43-96a1-60970009617c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 14 11:45:15.398219 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:15.398181 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:45:15.671749 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:15.671676 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:45:15.672433 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:45:15.672413 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:46:04.220122 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:04.220086 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl"] Jul 14 11:46:04.220669 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:04.220284 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" podUID="3e6b8635-e3ea-4b43-96a1-60970009617c" containerName="istio-proxy" containerID="cri-o://e33624dcd6981b1c37a676a3ec8389d754964f3bd2524da2486d77c00cb4f861" gracePeriod=30 Jul 14 11:46:04.250873 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:04.250836 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:46:04.362103 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:04.362025 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:46:09.471683 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.471658 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:46:09.593682 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.593650 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-envoy\") pod \"3e6b8635-e3ea-4b43-96a1-60970009617c\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " Jul 14 11:46:09.593802 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.593699 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-podinfo\") pod \"3e6b8635-e3ea-4b43-96a1-60970009617c\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " Jul 14 11:46:09.593802 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.593727 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-workload-certs\") pod \"3e6b8635-e3ea-4b43-96a1-60970009617c\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " Jul 14 11:46:09.593802 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.593769 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-credential-socket\") pod \"3e6b8635-e3ea-4b43-96a1-60970009617c\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " Jul 14 11:46:09.593935 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.593806 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-token\") pod \"3e6b8635-e3ea-4b43-96a1-60970009617c\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " Jul 14 11:46:09.593935 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.593829 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-workload-socket\") pod \"3e6b8635-e3ea-4b43-96a1-60970009617c\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " Jul 14 11:46:09.593935 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.593870 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6vc4\" (UniqueName: \"kubernetes.io/projected/3e6b8635-e3ea-4b43-96a1-60970009617c-kube-api-access-x6vc4\") pod \"3e6b8635-e3ea-4b43-96a1-60970009617c\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " Jul 14 11:46:09.594085 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.593966 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e6b8635-e3ea-4b43-96a1-60970009617c-istiod-ca-cert\") pod \"3e6b8635-e3ea-4b43-96a1-60970009617c\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " Jul 14 11:46:09.594085 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.594044 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3e6b8635-e3ea-4b43-96a1-60970009617c" (UID: "3e6b8635-e3ea-4b43-96a1-60970009617c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:46:09.594194 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.594071 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3e6b8635-e3ea-4b43-96a1-60970009617c" (UID: "3e6b8635-e3ea-4b43-96a1-60970009617c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:46:09.594194 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.594089 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-data\") pod \"3e6b8635-e3ea-4b43-96a1-60970009617c\" (UID: \"3e6b8635-e3ea-4b43-96a1-60970009617c\") " Jul 14 11:46:09.594410 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.594335 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3e6b8635-e3ea-4b43-96a1-60970009617c" (UID: "3e6b8635-e3ea-4b43-96a1-60970009617c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:46:09.594410 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.594393 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e6b8635-e3ea-4b43-96a1-60970009617c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3e6b8635-e3ea-4b43-96a1-60970009617c" (UID: "3e6b8635-e3ea-4b43-96a1-60970009617c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:46:09.594579 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.594470 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e6b8635-e3ea-4b43-96a1-60970009617c-istiod-ca-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:46:09.594579 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.594485 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-workload-certs\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:46:09.594579 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.594494 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-credential-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:46:09.594579 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.594505 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-workload-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:46:09.595046 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.595019 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-data" (OuterVolumeSpecName: "istio-data") pod "3e6b8635-e3ea-4b43-96a1-60970009617c" (UID: "3e6b8635-e3ea-4b43-96a1-60970009617c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:46:09.595959 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.595932 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3e6b8635-e3ea-4b43-96a1-60970009617c" (UID: "3e6b8635-e3ea-4b43-96a1-60970009617c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 11:46:09.596170 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.596143 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-token" (OuterVolumeSpecName: "istio-token") pod "3e6b8635-e3ea-4b43-96a1-60970009617c" (UID: "3e6b8635-e3ea-4b43-96a1-60970009617c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:46:09.596170 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.596143 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3e6b8635-e3ea-4b43-96a1-60970009617c" (UID: "3e6b8635-e3ea-4b43-96a1-60970009617c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:46:09.596287 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.596266 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e6b8635-e3ea-4b43-96a1-60970009617c-kube-api-access-x6vc4" (OuterVolumeSpecName: "kube-api-access-x6vc4") pod "3e6b8635-e3ea-4b43-96a1-60970009617c" (UID: "3e6b8635-e3ea-4b43-96a1-60970009617c"). InnerVolumeSpecName "kube-api-access-x6vc4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:46:09.695866 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.695839 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-data\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:46:09.695866 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.695861 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-envoy\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:46:09.695998 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.695870 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-podinfo\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:46:09.695998 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.695878 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e6b8635-e3ea-4b43-96a1-60970009617c-istio-token\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:46:09.695998 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.695888 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6vc4\" (UniqueName: \"kubernetes.io/projected/3e6b8635-e3ea-4b43-96a1-60970009617c-kube-api-access-x6vc4\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:46:09.773546 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.773509 2571 generic.go:356] "Generic (PLEG): container finished" podID="3e6b8635-e3ea-4b43-96a1-60970009617c" containerID="e33624dcd6981b1c37a676a3ec8389d754964f3bd2524da2486d77c00cb4f861" exitCode=0 Jul 14 11:46:09.773703 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.773564 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" Jul 14 11:46:09.773703 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.773593 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" event={"ID":"3e6b8635-e3ea-4b43-96a1-60970009617c","Type":"ContainerDied","Data":"e33624dcd6981b1c37a676a3ec8389d754964f3bd2524da2486d77c00cb4f861"} Jul 14 11:46:09.773703 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.773671 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl" event={"ID":"3e6b8635-e3ea-4b43-96a1-60970009617c","Type":"ContainerDied","Data":"d16216966248808e0f97c0c5ba2ac2dd4d8cd9381f6c056863401f9b64331857"} Jul 14 11:46:09.773703 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.773696 2571 scope.go:117] "RemoveContainer" containerID="e33624dcd6981b1c37a676a3ec8389d754964f3bd2524da2486d77c00cb4f861" Jul 14 11:46:09.783101 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.783082 2571 scope.go:117] "RemoveContainer" containerID="9cda09da178f7a6f351091f6d912f9e486c2ee5af34123a4bb9c6f73e07fc0dd" Jul 14 11:46:09.791002 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.790984 2571 scope.go:117] "RemoveContainer" containerID="e33624dcd6981b1c37a676a3ec8389d754964f3bd2524da2486d77c00cb4f861" Jul 14 11:46:09.791273 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:46:09.791255 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e33624dcd6981b1c37a676a3ec8389d754964f3bd2524da2486d77c00cb4f861\": container with ID starting with e33624dcd6981b1c37a676a3ec8389d754964f3bd2524da2486d77c00cb4f861 not found: ID does not exist" containerID="e33624dcd6981b1c37a676a3ec8389d754964f3bd2524da2486d77c00cb4f861" Jul 14 11:46:09.791319 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.791281 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e33624dcd6981b1c37a676a3ec8389d754964f3bd2524da2486d77c00cb4f861"} err="failed to get container status \"e33624dcd6981b1c37a676a3ec8389d754964f3bd2524da2486d77c00cb4f861\": rpc error: code = NotFound desc = could not find container \"e33624dcd6981b1c37a676a3ec8389d754964f3bd2524da2486d77c00cb4f861\": container with ID starting with e33624dcd6981b1c37a676a3ec8389d754964f3bd2524da2486d77c00cb4f861 not found: ID does not exist" Jul 14 11:46:09.791319 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.791300 2571 scope.go:117] "RemoveContainer" containerID="9cda09da178f7a6f351091f6d912f9e486c2ee5af34123a4bb9c6f73e07fc0dd" Jul 14 11:46:09.791533 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:46:09.791515 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cda09da178f7a6f351091f6d912f9e486c2ee5af34123a4bb9c6f73e07fc0dd\": container with ID starting with 9cda09da178f7a6f351091f6d912f9e486c2ee5af34123a4bb9c6f73e07fc0dd not found: ID does not exist" containerID="9cda09da178f7a6f351091f6d912f9e486c2ee5af34123a4bb9c6f73e07fc0dd" Jul 14 11:46:09.791574 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.791538 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cda09da178f7a6f351091f6d912f9e486c2ee5af34123a4bb9c6f73e07fc0dd"} err="failed to get container status \"9cda09da178f7a6f351091f6d912f9e486c2ee5af34123a4bb9c6f73e07fc0dd\": rpc error: code = NotFound desc = could not find container \"9cda09da178f7a6f351091f6d912f9e486c2ee5af34123a4bb9c6f73e07fc0dd\": container with ID starting with 9cda09da178f7a6f351091f6d912f9e486c2ee5af34123a4bb9c6f73e07fc0dd not found: ID does not exist" Jul 14 11:46:09.819830 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.819806 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl"] Jul 14 11:46:09.822661 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:09.822593 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-f22ceb3a-gw-openshift-default-55bd9849f-wzwbl"] Jul 14 11:46:10.098043 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:10.098012 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:46:10.404287 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:10.404194 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e6b8635-e3ea-4b43-96a1-60970009617c" path="/var/lib/kubelet/pods/3e6b8635-e3ea-4b43-96a1-60970009617c/volumes" Jul 14 11:46:10.638888 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:10.638854 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:46:10.772238 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:10.772205 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:46:16.413250 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.413216 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:46:16.476775 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.476747 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:46:16.641918 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.641891 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k"] Jul 14 11:46:16.642330 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.642313 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e6b8635-e3ea-4b43-96a1-60970009617c" containerName="istio-proxy" Jul 14 11:46:16.642421 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.642334 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e6b8635-e3ea-4b43-96a1-60970009617c" containerName="istio-proxy" Jul 14 11:46:16.642421 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.642349 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87a04dc7-74af-483e-b3b2-69146edbd593" containerName="istio-proxy" Jul 14 11:46:16.642421 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.642357 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="87a04dc7-74af-483e-b3b2-69146edbd593" containerName="istio-proxy" Jul 14 11:46:16.642567 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.642483 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="87a04dc7-74af-483e-b3b2-69146edbd593" containerName="istio-proxy" Jul 14 11:46:16.642567 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.642497 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e6b8635-e3ea-4b43-96a1-60970009617c" containerName="istio-proxy" Jul 14 11:46:16.642567 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.642510 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e6b8635-e3ea-4b43-96a1-60970009617c" containerName="istio-proxy" Jul 14 11:46:16.647096 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.647077 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.650674 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.650651 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4b103af8-openshift-default-dockercfg-ff8lr\"" Jul 14 11:46:16.661334 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.661315 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k"] Jul 14 11:46:16.681331 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.681273 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:46:16.761815 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.761772 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-credential-socket\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.761972 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.761824 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-token\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.761972 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.761904 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf4t5\" (UniqueName: \"kubernetes.io/projected/9cd38f55-bde5-4361-8a98-a39f090e1817-kube-api-access-vf4t5\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.761972 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.761941 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cd38f55-bde5-4361-8a98-a39f090e1817-istiod-ca-cert\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.761972 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.761970 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-workload-socket\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.762144 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.761989 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-workload-certs\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.762144 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.762074 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-data\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.762144 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.762098 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-podinfo\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.762144 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.762125 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-envoy\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.781310 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.781278 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:46:16.862824 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.862798 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-credential-socket\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.862992 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.862836 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-token\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.862992 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.862878 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf4t5\" (UniqueName: \"kubernetes.io/projected/9cd38f55-bde5-4361-8a98-a39f090e1817-kube-api-access-vf4t5\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.862992 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.862913 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cd38f55-bde5-4361-8a98-a39f090e1817-istiod-ca-cert\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.862992 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.862948 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-workload-socket\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.862992 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.862976 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-workload-certs\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.863263 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.863023 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-data\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.863263 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.863051 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-podinfo\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.863263 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.863087 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-envoy\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.863263 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.863249 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-credential-socket\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.863472 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.863348 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-workload-socket\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.863472 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.863434 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-workload-certs\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.863567 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.863494 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-data\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.863603 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.863581 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cd38f55-bde5-4361-8a98-a39f090e1817-istiod-ca-cert\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.865411 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.865389 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-envoy\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.865537 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.865519 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-podinfo\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.873979 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.873960 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-token\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.874293 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.874273 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf4t5\" (UniqueName: \"kubernetes.io/projected/9cd38f55-bde5-4361-8a98-a39f090e1817-kube-api-access-vf4t5\") pod \"e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:16.957227 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:16.957159 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:17.099203 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:17.099174 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k"] Jul 14 11:46:17.099364 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:46:17.099343 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cd38f55_bde5_4361_8a98_a39f090e1817.slice/crio-fe66b2588efae2313c21b8999bfa5aec728c77bb8ab48392f24fcc7eff682b47 WatchSource:0}: Error finding container fe66b2588efae2313c21b8999bfa5aec728c77bb8ab48392f24fcc7eff682b47: Status 404 returned error can't find the container with id fe66b2588efae2313c21b8999bfa5aec728c77bb8ab48392f24fcc7eff682b47 Jul 14 11:46:17.101619 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:17.101591 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:46:17.101704 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:17.101670 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:46:17.101704 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:17.101701 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:46:17.818529 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:17.818488 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" event={"ID":"9cd38f55-bde5-4361-8a98-a39f090e1817","Type":"ContainerStarted","Data":"1ea1f5a91cf3c3dfc358f1a892615e0acf6650896e85989832ebee3609fc6f08"} Jul 14 11:46:17.818931 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:17.818592 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" event={"ID":"9cd38f55-bde5-4361-8a98-a39f090e1817","Type":"ContainerStarted","Data":"fe66b2588efae2313c21b8999bfa5aec728c77bb8ab48392f24fcc7eff682b47"} Jul 14 11:46:17.851199 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:17.851152 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" podStartSLOduration=1.851136179 podStartE2EDuration="1.851136179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:46:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:46:17.848594628 +0000 UTC m=+1672.009671105" watchObservedRunningTime="2026-07-14 11:46:17.851136179 +0000 UTC m=+1672.012212646" Jul 14 11:46:17.883920 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:17.883865 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:46:17.957363 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:17.957333 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:17.994937 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:17.994901 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:46:18.087562 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:18.087426 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:46:18.220504 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:18.220475 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:46:18.660704 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:18.660652 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:46:18.762352 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:18.762303 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:46:18.962503 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:18.962431 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:19.830214 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:19.830187 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:19.831017 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:19.830995 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:19.975482 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:19.975448 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:46:28.880025 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:28.879988 2571 generic.go:356] "Generic (PLEG): container finished" podID="9cd38f55-bde5-4361-8a98-a39f090e1817" containerID="1ea1f5a91cf3c3dfc358f1a892615e0acf6650896e85989832ebee3609fc6f08" exitCode=0 Jul 14 11:46:28.880453 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:28.880057 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" event={"ID":"9cd38f55-bde5-4361-8a98-a39f090e1817","Type":"ContainerDied","Data":"1ea1f5a91cf3c3dfc358f1a892615e0acf6650896e85989832ebee3609fc6f08"} Jul 14 11:46:28.880453 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:28.880447 2571 scope.go:117] "RemoveContainer" containerID="1ea1f5a91cf3c3dfc358f1a892615e0acf6650896e85989832ebee3609fc6f08" Jul 14 11:46:28.881539 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:28.881514 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:46:28.881654 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:28.881576 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:46:28.881654 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:28.881606 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:46:29.890807 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:29.890765 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" event={"ID":"9cd38f55-bde5-4361-8a98-a39f090e1817","Type":"ContainerStarted","Data":"4644fe6c37489beedad142186065b0e2d5d1446085b39522ebdf4fc715a8e965"} Jul 14 11:46:29.957608 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:29.957573 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:29.957779 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:29.957762 2571 patch_prober.go:28] interesting pod/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 14 11:46:29.957823 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:29.957799 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" podUID="9cd38f55-bde5-4361-8a98-a39f090e1817" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 14 11:46:30.957573 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:30.957541 2571 patch_prober.go:28] interesting pod/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 14 11:46:30.957951 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:30.957599 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" podUID="9cd38f55-bde5-4361-8a98-a39f090e1817" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 14 11:46:31.958442 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:31.958405 2571 patch_prober.go:28] interesting pod/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 14 11:46:31.958861 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:31.958470 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" podUID="9cd38f55-bde5-4361-8a98-a39f090e1817" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 14 11:46:32.958271 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:32.958231 2571 patch_prober.go:28] interesting pod/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 14 11:46:32.958450 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:32.958294 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" podUID="9cd38f55-bde5-4361-8a98-a39f090e1817" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 14 11:46:33.961693 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:33.961664 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:33.962139 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:33.962039 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:33.962587 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:33.962571 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:46:52.174316 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:52.174225 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:46:52.224827 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:46:52.224788 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:47:03.592466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:03.592415 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:47:03.692495 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:03.692465 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:47:13.504608 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:13.504573 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:47:13.608891 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:13.608857 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:47:23.555679 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:23.555618 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:47:28.009725 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:28.009649 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:47:33.786198 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:33.786161 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:47:33.890974 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:33.890944 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:47:39.014850 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.014817 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k"] Jul 14 11:47:39.015207 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.015045 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" podUID="9cd38f55-bde5-4361-8a98-a39f090e1817" containerName="istio-proxy" containerID="cri-o://4644fe6c37489beedad142186065b0e2d5d1446085b39522ebdf4fc715a8e965" gracePeriod=30 Jul 14 11:47:39.037795 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.037770 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:47:39.307385 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.307313 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs"] Jul 14 11:47:39.307739 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.307726 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e6b8635-e3ea-4b43-96a1-60970009617c" containerName="istio-proxy" Jul 14 11:47:39.307791 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.307741 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e6b8635-e3ea-4b43-96a1-60970009617c" containerName="istio-proxy" Jul 14 11:47:39.307791 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.307751 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87a04dc7-74af-483e-b3b2-69146edbd593" containerName="istio-proxy" Jul 14 11:47:39.307791 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.307757 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="87a04dc7-74af-483e-b3b2-69146edbd593" containerName="istio-proxy" Jul 14 11:47:39.307881 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.307827 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="87a04dc7-74af-483e-b3b2-69146edbd593" containerName="istio-proxy" Jul 14 11:47:39.310997 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.310975 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.313931 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.313907 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-b8cbc777-openshift-default-dockercfg-m6ctm\"" Jul 14 11:47:39.324193 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.324172 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs"] Jul 14 11:47:39.343904 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.343857 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:47:39.379075 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.379053 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-workload-socket\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.379167 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.379114 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-credential-socket\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.379167 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.379132 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-envoy\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.379167 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.379151 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istiod-ca-cert\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.379297 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.379182 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkhnh\" (UniqueName: \"kubernetes.io/projected/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-kube-api-access-rkhnh\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.379297 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.379262 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-workload-certs\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.379297 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.379286 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-token\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.379408 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.379304 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-data\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.379408 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.379341 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-podinfo\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.480073 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.480039 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkhnh\" (UniqueName: \"kubernetes.io/projected/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-kube-api-access-rkhnh\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.480219 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.480079 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-workload-certs\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.480219 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.480098 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-token\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.480219 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.480115 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-data\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.480219 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.480157 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-podinfo\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.480219 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.480204 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-workload-socket\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.480483 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.480337 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-credential-socket\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.480483 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.480363 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-envoy\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.480483 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.480406 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istiod-ca-cert\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.480662 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.480505 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-workload-certs\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.480662 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.480556 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-data\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.480782 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.480721 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-credential-socket\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.480966 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.480942 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-workload-socket\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.481061 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.481041 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istiod-ca-cert\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.482578 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.482558 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-envoy\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.482701 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.482683 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-podinfo\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.488527 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.488507 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkhnh\" (UniqueName: \"kubernetes.io/projected/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-kube-api-access-rkhnh\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.488703 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.488683 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-token\") pod \"e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.623310 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.623290 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:39.756905 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.756880 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs"] Jul 14 11:47:39.758763 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:47:39.758734 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95a26f2d_c025_4d2a_9c17_a28149eb5fb1.slice/crio-c619437e47400066c394a8872dd1d402c92fbeb1fb4ee193a66ba6228f540e86 WatchSource:0}: Error finding container c619437e47400066c394a8872dd1d402c92fbeb1fb4ee193a66ba6228f540e86: Status 404 returned error can't find the container with id c619437e47400066c394a8872dd1d402c92fbeb1fb4ee193a66ba6228f540e86 Jul 14 11:47:39.760785 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.760752 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:47:39.760893 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.760818 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:47:39.760893 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:39.760848 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:47:40.264712 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:40.264673 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" event={"ID":"95a26f2d-c025-4d2a-9c17-a28149eb5fb1","Type":"ContainerStarted","Data":"8e80bd841732bd83af52613d319e1aa68e3a8970db60d0b6d3627b49f1d056f8"} Jul 14 11:47:40.265075 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:40.264739 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" event={"ID":"95a26f2d-c025-4d2a-9c17-a28149eb5fb1","Type":"ContainerStarted","Data":"c619437e47400066c394a8872dd1d402c92fbeb1fb4ee193a66ba6228f540e86"} Jul 14 11:47:40.287127 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:40.287071 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" podStartSLOduration=1.287052863 podStartE2EDuration="1.287052863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:47:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:47:40.286350867 +0000 UTC m=+1754.447427336" watchObservedRunningTime="2026-07-14 11:47:40.287052863 +0000 UTC m=+1754.448129329" Jul 14 11:47:40.530500 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:40.530424 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:47:40.623941 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:40.623914 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:40.628741 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:40.628714 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:40.630513 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:40.630484 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:47:40.774537 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:40.774500 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:47:41.088428 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:41.088395 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:47:41.277185 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:41.277127 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:41.278223 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:41.278203 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:41.361424 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:41.361392 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:47:41.464223 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:41.464195 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:47:44.299962 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.299874 2571 generic.go:356] "Generic (PLEG): container finished" podID="9cd38f55-bde5-4361-8a98-a39f090e1817" containerID="4644fe6c37489beedad142186065b0e2d5d1446085b39522ebdf4fc715a8e965" exitCode=0 Jul 14 11:47:44.300389 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.299954 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" event={"ID":"9cd38f55-bde5-4361-8a98-a39f090e1817","Type":"ContainerDied","Data":"4644fe6c37489beedad142186065b0e2d5d1446085b39522ebdf4fc715a8e965"} Jul 14 11:47:44.300389 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.300038 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" event={"ID":"9cd38f55-bde5-4361-8a98-a39f090e1817","Type":"ContainerDied","Data":"fe66b2588efae2313c21b8999bfa5aec728c77bb8ab48392f24fcc7eff682b47"} Jul 14 11:47:44.300389 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.300067 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe66b2588efae2313c21b8999bfa5aec728c77bb8ab48392f24fcc7eff682b47" Jul 14 11:47:44.300389 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.300089 2571 scope.go:117] "RemoveContainer" containerID="1ea1f5a91cf3c3dfc358f1a892615e0acf6650896e85989832ebee3609fc6f08" Jul 14 11:47:44.304498 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.304479 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:47:44.425858 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.425832 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cd38f55-bde5-4361-8a98-a39f090e1817-istiod-ca-cert\") pod \"9cd38f55-bde5-4361-8a98-a39f090e1817\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " Jul 14 11:47:44.425973 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.425879 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-data\") pod \"9cd38f55-bde5-4361-8a98-a39f090e1817\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " Jul 14 11:47:44.425973 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.425899 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-workload-certs\") pod \"9cd38f55-bde5-4361-8a98-a39f090e1817\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " Jul 14 11:47:44.425973 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.425925 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-workload-socket\") pod \"9cd38f55-bde5-4361-8a98-a39f090e1817\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " Jul 14 11:47:44.425973 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.425961 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-envoy\") pod \"9cd38f55-bde5-4361-8a98-a39f090e1817\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " Jul 14 11:47:44.426160 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.425990 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-token\") pod \"9cd38f55-bde5-4361-8a98-a39f090e1817\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " Jul 14 11:47:44.426160 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.426039 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vf4t5\" (UniqueName: \"kubernetes.io/projected/9cd38f55-bde5-4361-8a98-a39f090e1817-kube-api-access-vf4t5\") pod \"9cd38f55-bde5-4361-8a98-a39f090e1817\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " Jul 14 11:47:44.426160 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.426114 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-podinfo\") pod \"9cd38f55-bde5-4361-8a98-a39f090e1817\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " Jul 14 11:47:44.426160 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.426157 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-credential-socket\") pod \"9cd38f55-bde5-4361-8a98-a39f090e1817\" (UID: \"9cd38f55-bde5-4361-8a98-a39f090e1817\") " Jul 14 11:47:44.426380 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.426208 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9cd38f55-bde5-4361-8a98-a39f090e1817" (UID: "9cd38f55-bde5-4361-8a98-a39f090e1817"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:47:44.426380 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.426222 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9cd38f55-bde5-4361-8a98-a39f090e1817" (UID: "9cd38f55-bde5-4361-8a98-a39f090e1817"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:47:44.426380 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.426265 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cd38f55-bde5-4361-8a98-a39f090e1817-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9cd38f55-bde5-4361-8a98-a39f090e1817" (UID: "9cd38f55-bde5-4361-8a98-a39f090e1817"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:47:44.426545 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.426476 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9cd38f55-bde5-4361-8a98-a39f090e1817" (UID: "9cd38f55-bde5-4361-8a98-a39f090e1817"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:47:44.426602 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.426583 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cd38f55-bde5-4361-8a98-a39f090e1817-istiod-ca-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:47:44.426602 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.426599 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-workload-certs\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:47:44.426724 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.426611 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-workload-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:47:44.426724 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.426625 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-credential-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:47:44.426948 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.426921 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-data" (OuterVolumeSpecName: "istio-data") pod "9cd38f55-bde5-4361-8a98-a39f090e1817" (UID: "9cd38f55-bde5-4361-8a98-a39f090e1817"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:47:44.428153 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.428122 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9cd38f55-bde5-4361-8a98-a39f090e1817" (UID: "9cd38f55-bde5-4361-8a98-a39f090e1817"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:47:44.428259 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.428144 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-token" (OuterVolumeSpecName: "istio-token") pod "9cd38f55-bde5-4361-8a98-a39f090e1817" (UID: "9cd38f55-bde5-4361-8a98-a39f090e1817"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:47:44.428259 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.428165 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cd38f55-bde5-4361-8a98-a39f090e1817-kube-api-access-vf4t5" (OuterVolumeSpecName: "kube-api-access-vf4t5") pod "9cd38f55-bde5-4361-8a98-a39f090e1817" (UID: "9cd38f55-bde5-4361-8a98-a39f090e1817"). InnerVolumeSpecName "kube-api-access-vf4t5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:47:44.428259 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.428237 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9cd38f55-bde5-4361-8a98-a39f090e1817" (UID: "9cd38f55-bde5-4361-8a98-a39f090e1817"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 11:47:44.529270 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.529231 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-podinfo\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:47:44.529270 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.529271 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-data\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:47:44.529527 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.529283 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-envoy\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:47:44.529527 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.529296 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cd38f55-bde5-4361-8a98-a39f090e1817-istio-token\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:47:44.529527 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:44.529308 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vf4t5\" (UniqueName: \"kubernetes.io/projected/9cd38f55-bde5-4361-8a98-a39f090e1817-kube-api-access-vf4t5\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:47:45.307016 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:45.306987 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k" Jul 14 11:47:45.373060 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:45.373026 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k"] Jul 14 11:47:45.376809 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:45.376780 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4b103af8-openshift-default-5f4647d997-d4r8k"] Jul 14 11:47:46.403914 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:46.403883 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cd38f55-bde5-4361-8a98-a39f090e1817" path="/var/lib/kubelet/pods/9cd38f55-bde5-4361-8a98-a39f090e1817/volumes" Jul 14 11:47:51.342208 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:51.342179 2571 generic.go:356] "Generic (PLEG): container finished" podID="95a26f2d-c025-4d2a-9c17-a28149eb5fb1" containerID="8e80bd841732bd83af52613d319e1aa68e3a8970db60d0b6d3627b49f1d056f8" exitCode=0 Jul 14 11:47:51.342592 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:51.342213 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" event={"ID":"95a26f2d-c025-4d2a-9c17-a28149eb5fb1","Type":"ContainerDied","Data":"8e80bd841732bd83af52613d319e1aa68e3a8970db60d0b6d3627b49f1d056f8"} Jul 14 11:47:51.342592 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:51.342578 2571 scope.go:117] "RemoveContainer" containerID="8e80bd841732bd83af52613d319e1aa68e3a8970db60d0b6d3627b49f1d056f8" Jul 14 11:47:51.343487 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:51.343462 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:47:51.343538 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:51.343527 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:47:51.343574 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:51.343560 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:47:52.348846 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:52.348814 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" event={"ID":"95a26f2d-c025-4d2a-9c17-a28149eb5fb1","Type":"ContainerStarted","Data":"091b4ce63693b310f564982dd82504724bfad1ae260b8166fe5733b9b5d589fe"} Jul 14 11:47:52.624014 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:52.623917 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:52.625198 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:52.625175 2571 patch_prober.go:28] interesting pod/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 14 11:47:52.625260 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:52.625239 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" podUID="95a26f2d-c025-4d2a-9c17-a28149eb5fb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 14 11:47:53.624264 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:53.624232 2571 patch_prober.go:28] interesting pod/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 14 11:47:53.624623 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:53.624299 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" podUID="95a26f2d-c025-4d2a-9c17-a28149eb5fb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 14 11:47:54.624712 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:54.624680 2571 patch_prober.go:28] interesting pod/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 14 11:47:54.625063 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:54.624749 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" podUID="95a26f2d-c025-4d2a-9c17-a28149eb5fb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 14 11:47:55.495284 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:55.495250 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:47:55.594521 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:55.594469 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:47:55.624679 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:55.624649 2571 patch_prober.go:28] interesting pod/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 14 11:47:55.624816 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:55.624701 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" podUID="95a26f2d-c025-4d2a-9c17-a28149eb5fb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 14 11:47:56.277443 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:56.277412 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:56.627707 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:56.627683 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:56.629478 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:56.629454 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:47:59.054186 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:47:59.054157 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:48:26.447165 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:48:26.447138 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 11:48:26.454937 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:48:26.454917 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 11:48:55.453009 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:48:55.452971 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:48:59.993788 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:48:59.993758 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:49:05.769784 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:05.769755 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:49:05.834035 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:05.834009 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:49:10.176645 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:10.176606 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs"] Jul 14 11:49:10.177044 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:10.176814 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" podUID="95a26f2d-c025-4d2a-9c17-a28149eb5fb1" containerName="istio-proxy" containerID="cri-o://091b4ce63693b310f564982dd82504724bfad1ae260b8166fe5733b9b5d589fe" gracePeriod=30 Jul 14 11:49:10.200560 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:10.200522 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:49:10.794522 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:10.794488 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:49:10.894890 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:10.894860 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:49:10.994181 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:10.994156 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:49:11.630483 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:11.630453 2571 patch_prober.go:28] interesting pod/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 11:49:11.630826 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:11.630505 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" podUID="95a26f2d-c025-4d2a-9c17-a28149eb5fb1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 11:49:12.471984 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:12.471950 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:49:15.437262 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.437236 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:49:15.546561 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.546535 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkhnh\" (UniqueName: \"kubernetes.io/projected/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-kube-api-access-rkhnh\") pod \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " Jul 14 11:49:15.546691 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.546573 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-workload-certs\") pod \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " Jul 14 11:49:15.546691 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.546595 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istiod-ca-cert\") pod \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " Jul 14 11:49:15.546691 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.546613 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-podinfo\") pod \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " Jul 14 11:49:15.546691 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.546652 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-credential-socket\") pod \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " Jul 14 11:49:15.546691 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.546675 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-data\") pod \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " Jul 14 11:49:15.546935 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.546707 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-envoy\") pod \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " Jul 14 11:49:15.546935 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.546739 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-token\") pod \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " Jul 14 11:49:15.546935 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.546769 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-workload-socket\") pod \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\" (UID: \"95a26f2d-c025-4d2a-9c17-a28149eb5fb1\") " Jul 14 11:49:15.546935 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.546870 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "95a26f2d-c025-4d2a-9c17-a28149eb5fb1" (UID: "95a26f2d-c025-4d2a-9c17-a28149eb5fb1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:49:15.547139 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.546964 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "95a26f2d-c025-4d2a-9c17-a28149eb5fb1" (UID: "95a26f2d-c025-4d2a-9c17-a28149eb5fb1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:49:15.547139 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.547014 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "95a26f2d-c025-4d2a-9c17-a28149eb5fb1" (UID: "95a26f2d-c025-4d2a-9c17-a28149eb5fb1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:49:15.547246 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.547128 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "95a26f2d-c025-4d2a-9c17-a28149eb5fb1" (UID: "95a26f2d-c025-4d2a-9c17-a28149eb5fb1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:49:15.547296 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.547253 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-workload-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:49:15.547296 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.547273 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-workload-certs\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:49:15.547296 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.547287 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istiod-ca-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:49:15.547442 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.547300 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-credential-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:49:15.547871 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.547841 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-data" (OuterVolumeSpecName: "istio-data") pod "95a26f2d-c025-4d2a-9c17-a28149eb5fb1" (UID: "95a26f2d-c025-4d2a-9c17-a28149eb5fb1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:49:15.548717 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.548686 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "95a26f2d-c025-4d2a-9c17-a28149eb5fb1" (UID: "95a26f2d-c025-4d2a-9c17-a28149eb5fb1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:49:15.548808 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.548713 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-kube-api-access-rkhnh" (OuterVolumeSpecName: "kube-api-access-rkhnh") pod "95a26f2d-c025-4d2a-9c17-a28149eb5fb1" (UID: "95a26f2d-c025-4d2a-9c17-a28149eb5fb1"). InnerVolumeSpecName "kube-api-access-rkhnh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:49:15.548880 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.548823 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-token" (OuterVolumeSpecName: "istio-token") pod "95a26f2d-c025-4d2a-9c17-a28149eb5fb1" (UID: "95a26f2d-c025-4d2a-9c17-a28149eb5fb1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:49:15.548967 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.548950 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "95a26f2d-c025-4d2a-9c17-a28149eb5fb1" (UID: "95a26f2d-c025-4d2a-9c17-a28149eb5fb1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 11:49:15.648296 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.648271 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkhnh\" (UniqueName: \"kubernetes.io/projected/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-kube-api-access-rkhnh\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:49:15.648296 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.648295 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-podinfo\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:49:15.648423 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.648304 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-data\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:49:15.648423 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.648312 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-envoy\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:49:15.648423 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.648319 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/95a26f2d-c025-4d2a-9c17-a28149eb5fb1-istio-token\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:49:15.785616 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.785554 2571 generic.go:356] "Generic (PLEG): container finished" podID="95a26f2d-c025-4d2a-9c17-a28149eb5fb1" containerID="091b4ce63693b310f564982dd82504724bfad1ae260b8166fe5733b9b5d589fe" exitCode=0 Jul 14 11:49:15.785616 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.785599 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" Jul 14 11:49:15.785775 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.785662 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" event={"ID":"95a26f2d-c025-4d2a-9c17-a28149eb5fb1","Type":"ContainerDied","Data":"091b4ce63693b310f564982dd82504724bfad1ae260b8166fe5733b9b5d589fe"} Jul 14 11:49:15.785775 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.785737 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs" event={"ID":"95a26f2d-c025-4d2a-9c17-a28149eb5fb1","Type":"ContainerDied","Data":"c619437e47400066c394a8872dd1d402c92fbeb1fb4ee193a66ba6228f540e86"} Jul 14 11:49:15.785775 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.785759 2571 scope.go:117] "RemoveContainer" containerID="091b4ce63693b310f564982dd82504724bfad1ae260b8166fe5733b9b5d589fe" Jul 14 11:49:15.794896 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.794882 2571 scope.go:117] "RemoveContainer" containerID="8e80bd841732bd83af52613d319e1aa68e3a8970db60d0b6d3627b49f1d056f8" Jul 14 11:49:15.802527 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.802513 2571 scope.go:117] "RemoveContainer" containerID="091b4ce63693b310f564982dd82504724bfad1ae260b8166fe5733b9b5d589fe" Jul 14 11:49:15.802768 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:49:15.802748 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"091b4ce63693b310f564982dd82504724bfad1ae260b8166fe5733b9b5d589fe\": container with ID starting with 091b4ce63693b310f564982dd82504724bfad1ae260b8166fe5733b9b5d589fe not found: ID does not exist" containerID="091b4ce63693b310f564982dd82504724bfad1ae260b8166fe5733b9b5d589fe" Jul 14 11:49:15.802847 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.802779 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"091b4ce63693b310f564982dd82504724bfad1ae260b8166fe5733b9b5d589fe"} err="failed to get container status \"091b4ce63693b310f564982dd82504724bfad1ae260b8166fe5733b9b5d589fe\": rpc error: code = NotFound desc = could not find container \"091b4ce63693b310f564982dd82504724bfad1ae260b8166fe5733b9b5d589fe\": container with ID starting with 091b4ce63693b310f564982dd82504724bfad1ae260b8166fe5733b9b5d589fe not found: ID does not exist" Jul 14 11:49:15.802847 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.802801 2571 scope.go:117] "RemoveContainer" containerID="8e80bd841732bd83af52613d319e1aa68e3a8970db60d0b6d3627b49f1d056f8" Jul 14 11:49:15.803034 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:49:15.803017 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e80bd841732bd83af52613d319e1aa68e3a8970db60d0b6d3627b49f1d056f8\": container with ID starting with 8e80bd841732bd83af52613d319e1aa68e3a8970db60d0b6d3627b49f1d056f8 not found: ID does not exist" containerID="8e80bd841732bd83af52613d319e1aa68e3a8970db60d0b6d3627b49f1d056f8" Jul 14 11:49:15.803072 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.803044 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e80bd841732bd83af52613d319e1aa68e3a8970db60d0b6d3627b49f1d056f8"} err="failed to get container status \"8e80bd841732bd83af52613d319e1aa68e3a8970db60d0b6d3627b49f1d056f8\": rpc error: code = NotFound desc = could not find container \"8e80bd841732bd83af52613d319e1aa68e3a8970db60d0b6d3627b49f1d056f8\": container with ID starting with 8e80bd841732bd83af52613d319e1aa68e3a8970db60d0b6d3627b49f1d056f8 not found: ID does not exist" Jul 14 11:49:15.808621 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.808600 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs"] Jul 14 11:49:15.812121 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:15.812099 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-b8cbc777-openshift-default-774fbdc77f-s7nfs"] Jul 14 11:49:16.403917 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:16.403883 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95a26f2d-c025-4d2a-9c17-a28149eb5fb1" path="/var/lib/kubelet/pods/95a26f2d-c025-4d2a-9c17-a28149eb5fb1/volumes" Jul 14 11:49:16.482204 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:16.482172 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:49:18.526643 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:18.526604 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:49:18.590444 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:18.590418 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:49:18.676576 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:18.676545 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:49:18.788958 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:18.788887 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:49:18.923853 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:18.923825 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:49:19.291977 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:19.291952 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:49:45.254508 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:45.254479 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:49:51.546361 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:51.546286 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:49:51.717747 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:51.717714 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:49:51.817219 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:49:51.817142 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:07.675996 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:07.675960 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:13.585825 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:13.585791 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:13.657532 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:13.657489 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:13.788052 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:13.788026 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:13.872984 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:13.872942 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:19.930619 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:19.930592 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:24.349340 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:24.349302 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:25.193365 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:25.193332 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:31.092380 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:31.092348 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:31.167448 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:31.167415 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:31.254810 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:31.254774 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:31.354840 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:31.354811 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:31.454250 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:31.454228 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:32.241281 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:32.241250 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:32.448117 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:32.448076 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:34.076684 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:34.076653 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:40.076580 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:40.076547 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:40.176490 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:40.176465 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:40.761508 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:40.761470 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:46.889212 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:46.889176 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:46.988998 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:46.988972 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:47.155915 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.155846 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:47.452553 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.452479 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns"] Jul 14 11:50:47.452940 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.452921 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cd38f55-bde5-4361-8a98-a39f090e1817" containerName="istio-proxy" Jul 14 11:50:47.453031 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.452942 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cd38f55-bde5-4361-8a98-a39f090e1817" containerName="istio-proxy" Jul 14 11:50:47.453031 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.452974 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95a26f2d-c025-4d2a-9c17-a28149eb5fb1" containerName="istio-proxy" Jul 14 11:50:47.453031 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.452983 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="95a26f2d-c025-4d2a-9c17-a28149eb5fb1" containerName="istio-proxy" Jul 14 11:50:47.453031 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.452994 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95a26f2d-c025-4d2a-9c17-a28149eb5fb1" containerName="istio-proxy" Jul 14 11:50:47.453031 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.453003 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="95a26f2d-c025-4d2a-9c17-a28149eb5fb1" containerName="istio-proxy" Jul 14 11:50:47.453283 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.453107 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cd38f55-bde5-4361-8a98-a39f090e1817" containerName="istio-proxy" Jul 14 11:50:47.453283 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.453120 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="95a26f2d-c025-4d2a-9c17-a28149eb5fb1" containerName="istio-proxy" Jul 14 11:50:47.453283 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.453130 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="95a26f2d-c025-4d2a-9c17-a28149eb5fb1" containerName="istio-proxy" Jul 14 11:50:47.453283 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.453141 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cd38f55-bde5-4361-8a98-a39f090e1817" containerName="istio-proxy" Jul 14 11:50:47.456248 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.456226 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.459063 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.459046 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-3f0f93e1-openshift-default-dockercfg-4lnvk\"" Jul 14 11:50:47.474949 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.474904 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:47.476462 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.476440 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns"] Jul 14 11:50:47.555394 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.555357 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl4df\" (UniqueName: \"kubernetes.io/projected/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-kube-api-access-tl4df\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.555578 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.555419 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-data\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.555578 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.555500 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-credential-socket\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.555578 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.555522 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-workload-socket\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.555578 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.555553 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-envoy\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.555844 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.555612 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-podinfo\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.555844 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.555655 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-workload-certs\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.555844 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.555682 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-token\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.555844 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.555720 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istiod-ca-cert\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.656287 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.656261 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-podinfo\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.656419 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.656290 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-workload-certs\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.656419 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.656309 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-token\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.656419 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.656327 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istiod-ca-cert\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.656419 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.656364 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl4df\" (UniqueName: \"kubernetes.io/projected/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-kube-api-access-tl4df\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.656419 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.656386 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-data\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.656419 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.656413 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-credential-socket\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.656745 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.656427 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-workload-socket\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.656745 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.656449 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-envoy\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.656745 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.656726 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-workload-certs\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.656897 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.656810 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-credential-socket\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.656897 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.656862 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-workload-socket\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.657007 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.656991 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-data\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.657007 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.657000 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istiod-ca-cert\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.658681 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.658662 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-envoy\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.658884 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.658868 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-podinfo\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.665425 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.665398 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-token\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.665595 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.665547 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl4df\" (UniqueName: \"kubernetes.io/projected/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-kube-api-access-tl4df\") pod \"e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.766589 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.766542 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:47.866998 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.866951 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:47.907505 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.907475 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns"] Jul 14 11:50:47.907835 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:50:47.907690 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod684ea306_42c2_4c9d_a1ba_0e9a17dcd5af.slice/crio-44c6a46edaba2c97b09cb777dc46800b6c9dabe56accbdf60c5f1f62e66f7e3d WatchSource:0}: Error finding container 44c6a46edaba2c97b09cb777dc46800b6c9dabe56accbdf60c5f1f62e66f7e3d: Status 404 returned error can't find the container with id 44c6a46edaba2c97b09cb777dc46800b6c9dabe56accbdf60c5f1f62e66f7e3d Jul 14 11:50:47.910046 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.910011 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:50:47.910133 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.910100 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:50:47.910171 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:47.910140 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:50:48.262126 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:48.262087 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" event={"ID":"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af","Type":"ContainerStarted","Data":"c89dbfd756375ef9a53d0a079ef20d24725b1f2c41c458ee5bc821ade130a432"} Jul 14 11:50:48.262296 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:48.262182 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" event={"ID":"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af","Type":"ContainerStarted","Data":"44c6a46edaba2c97b09cb777dc46800b6c9dabe56accbdf60c5f1f62e66f7e3d"} Jul 14 11:50:48.286750 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:48.286706 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" podStartSLOduration=1.2866918 podStartE2EDuration="1.2866918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:50:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:50:48.284247414 +0000 UTC m=+1942.445323892" watchObservedRunningTime="2026-07-14 11:50:48.2866918 +0000 UTC m=+1942.447768265" Jul 14 11:50:48.767696 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:48.767664 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:48.772155 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:48.772132 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:49.266770 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:49.266739 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:49.267609 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:49.267590 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:50:50.249795 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:50.249744 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:50.343413 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:50.343367 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:50.439881 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:50.439853 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:51.109447 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:51.109417 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:51.213455 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:51.213427 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:54.733489 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:54.733459 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:55.411118 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:55.411087 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:55.507786 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:55.507742 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:50:55.841901 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:50:55.841805 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:01.851505 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:01.851459 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:02.008683 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:02.008654 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:03.351072 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:03.351046 2571 generic.go:356] "Generic (PLEG): container finished" podID="684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" containerID="c89dbfd756375ef9a53d0a079ef20d24725b1f2c41c458ee5bc821ade130a432" exitCode=0 Jul 14 11:51:03.351433 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:03.351119 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" event={"ID":"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af","Type":"ContainerDied","Data":"c89dbfd756375ef9a53d0a079ef20d24725b1f2c41c458ee5bc821ade130a432"} Jul 14 11:51:03.351485 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:03.351471 2571 scope.go:117] "RemoveContainer" containerID="c89dbfd756375ef9a53d0a079ef20d24725b1f2c41c458ee5bc821ade130a432" Jul 14 11:51:03.352557 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:03.352529 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:51:03.352642 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:03.352612 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:51:03.352690 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:03.352681 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:51:04.266738 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:04.266706 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:51:04.384304 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:04.384257 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" event={"ID":"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af","Type":"ContainerStarted","Data":"620fe1cd81743dacadafe2fbf8f230fd2de44c86180f11dd3a92468756094fda"} Jul 14 11:51:04.767503 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:04.767462 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:51:04.768587 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:04.768556 2571 patch_prober.go:28] interesting pod/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 14 11:51:04.768714 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:04.768663 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" podUID="684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 14 11:51:05.767472 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:05.767429 2571 patch_prober.go:28] interesting pod/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 14 11:51:05.767961 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:05.767510 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" podUID="684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 14 11:51:06.767085 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:06.767048 2571 patch_prober.go:28] interesting pod/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 14 11:51:06.767277 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:06.767107 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" podUID="684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 14 11:51:07.767008 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:07.766966 2571 patch_prober.go:28] interesting pod/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 14 11:51:07.767476 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:07.767028 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" podUID="684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 14 11:51:08.770580 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:08.770552 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:51:08.770970 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:08.770904 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:51:08.771511 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:08.771494 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:51:12.846741 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:12.846707 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:17.308317 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:17.308285 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:23.452146 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:23.452060 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:23.533123 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:23.533089 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:28.018539 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:28.018501 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns"] Jul 14 11:51:28.018932 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:28.018765 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" podUID="684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" containerName="istio-proxy" containerID="cri-o://620fe1cd81743dacadafe2fbf8f230fd2de44c86180f11dd3a92468756094fda" gracePeriod=30 Jul 14 11:51:28.035364 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:28.035335 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:28.255405 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:28.255376 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:28.396481 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:28.396454 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:28.488878 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:28.488850 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:28.599164 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:28.599132 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:33.265000 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.264979 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:51:33.328420 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.328351 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-podinfo\") pod \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " Jul 14 11:51:33.328420 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.328388 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-workload-certs\") pod \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " Jul 14 11:51:33.328586 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.328443 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-data\") pod \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " Jul 14 11:51:33.328586 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.328495 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-workload-socket\") pod \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " Jul 14 11:51:33.328586 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.328522 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istiod-ca-cert\") pod \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " Jul 14 11:51:33.328586 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.328559 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-envoy\") pod \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " Jul 14 11:51:33.328819 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.328618 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-credential-socket\") pod \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " Jul 14 11:51:33.328819 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.328771 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" (UID: "684ea306-42c2-4c9d-a1ba-0e9a17dcd5af"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:51:33.328819 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.328784 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-token\") pod \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " Jul 14 11:51:33.328975 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.328861 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tl4df\" (UniqueName: \"kubernetes.io/projected/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-kube-api-access-tl4df\") pod \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\" (UID: \"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af\") " Jul 14 11:51:33.329031 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.328972 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" (UID: "684ea306-42c2-4c9d-a1ba-0e9a17dcd5af"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:51:33.329031 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.328987 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" (UID: "684ea306-42c2-4c9d-a1ba-0e9a17dcd5af"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:51:33.329188 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.329165 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" (UID: "684ea306-42c2-4c9d-a1ba-0e9a17dcd5af"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:51:33.329293 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.329278 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-workload-certs\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:51:33.329353 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.329301 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-workload-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:51:33.329353 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.329314 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istiod-ca-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:51:33.329353 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.329327 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-credential-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:51:33.329463 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.329365 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-data" (OuterVolumeSpecName: "istio-data") pod "684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" (UID: "684ea306-42c2-4c9d-a1ba-0e9a17dcd5af"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:51:33.330791 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.330762 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" (UID: "684ea306-42c2-4c9d-a1ba-0e9a17dcd5af"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:51:33.330881 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.330852 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-token" (OuterVolumeSpecName: "istio-token") pod "684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" (UID: "684ea306-42c2-4c9d-a1ba-0e9a17dcd5af"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:51:33.330936 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.330921 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" (UID: "684ea306-42c2-4c9d-a1ba-0e9a17dcd5af"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 11:51:33.331136 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.331117 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-kube-api-access-tl4df" (OuterVolumeSpecName: "kube-api-access-tl4df") pod "684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" (UID: "684ea306-42c2-4c9d-a1ba-0e9a17dcd5af"). InnerVolumeSpecName "kube-api-access-tl4df". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:51:33.430535 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.430509 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-envoy\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:51:33.430535 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.430531 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-token\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:51:33.430695 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.430544 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tl4df\" (UniqueName: \"kubernetes.io/projected/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-kube-api-access-tl4df\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:51:33.430695 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.430556 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-podinfo\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:51:33.430695 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.430568 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af-istio-data\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:51:33.530786 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.530759 2571 generic.go:356] "Generic (PLEG): container finished" podID="684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" containerID="620fe1cd81743dacadafe2fbf8f230fd2de44c86180f11dd3a92468756094fda" exitCode=0 Jul 14 11:51:33.530872 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.530801 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" Jul 14 11:51:33.530872 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.530845 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" event={"ID":"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af","Type":"ContainerDied","Data":"620fe1cd81743dacadafe2fbf8f230fd2de44c86180f11dd3a92468756094fda"} Jul 14 11:51:33.530943 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.530900 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns" event={"ID":"684ea306-42c2-4c9d-a1ba-0e9a17dcd5af","Type":"ContainerDied","Data":"44c6a46edaba2c97b09cb777dc46800b6c9dabe56accbdf60c5f1f62e66f7e3d"} Jul 14 11:51:33.530943 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.530928 2571 scope.go:117] "RemoveContainer" containerID="620fe1cd81743dacadafe2fbf8f230fd2de44c86180f11dd3a92468756094fda" Jul 14 11:51:33.540253 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.540238 2571 scope.go:117] "RemoveContainer" containerID="c89dbfd756375ef9a53d0a079ef20d24725b1f2c41c458ee5bc821ade130a432" Jul 14 11:51:33.548212 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.548194 2571 scope.go:117] "RemoveContainer" containerID="620fe1cd81743dacadafe2fbf8f230fd2de44c86180f11dd3a92468756094fda" Jul 14 11:51:33.548501 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:51:33.548483 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"620fe1cd81743dacadafe2fbf8f230fd2de44c86180f11dd3a92468756094fda\": container with ID starting with 620fe1cd81743dacadafe2fbf8f230fd2de44c86180f11dd3a92468756094fda not found: ID does not exist" containerID="620fe1cd81743dacadafe2fbf8f230fd2de44c86180f11dd3a92468756094fda" Jul 14 11:51:33.548571 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.548507 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"620fe1cd81743dacadafe2fbf8f230fd2de44c86180f11dd3a92468756094fda"} err="failed to get container status \"620fe1cd81743dacadafe2fbf8f230fd2de44c86180f11dd3a92468756094fda\": rpc error: code = NotFound desc = could not find container \"620fe1cd81743dacadafe2fbf8f230fd2de44c86180f11dd3a92468756094fda\": container with ID starting with 620fe1cd81743dacadafe2fbf8f230fd2de44c86180f11dd3a92468756094fda not found: ID does not exist" Jul 14 11:51:33.548571 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.548523 2571 scope.go:117] "RemoveContainer" containerID="c89dbfd756375ef9a53d0a079ef20d24725b1f2c41c458ee5bc821ade130a432" Jul 14 11:51:33.548824 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:51:33.548810 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c89dbfd756375ef9a53d0a079ef20d24725b1f2c41c458ee5bc821ade130a432\": container with ID starting with c89dbfd756375ef9a53d0a079ef20d24725b1f2c41c458ee5bc821ade130a432 not found: ID does not exist" containerID="c89dbfd756375ef9a53d0a079ef20d24725b1f2c41c458ee5bc821ade130a432" Jul 14 11:51:33.548865 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.548829 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c89dbfd756375ef9a53d0a079ef20d24725b1f2c41c458ee5bc821ade130a432"} err="failed to get container status \"c89dbfd756375ef9a53d0a079ef20d24725b1f2c41c458ee5bc821ade130a432\": rpc error: code = NotFound desc = could not find container \"c89dbfd756375ef9a53d0a079ef20d24725b1f2c41c458ee5bc821ade130a432\": container with ID starting with c89dbfd756375ef9a53d0a079ef20d24725b1f2c41c458ee5bc821ade130a432 not found: ID does not exist" Jul 14 11:51:33.555778 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.555757 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns"] Jul 14 11:51:33.560834 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:33.560814 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-3f0f93e1-openshift-default-77d957ddc9-n8fns"] Jul 14 11:51:34.403345 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:34.403316 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" path="/var/lib/kubelet/pods/684ea306-42c2-4c9d-a1ba-0e9a17dcd5af/volumes" Jul 14 11:51:34.722040 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:34.721963 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:34.832082 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:34.832054 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:34.925016 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:34.924956 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:35.057582 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:35.057508 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:40.824091 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:40.824058 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:40.924696 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:40.924647 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:47.631152 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:47.631110 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:47.925602 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:47.925524 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8"] Jul 14 11:51:47.925969 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:47.925953 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" containerName="istio-proxy" Jul 14 11:51:47.926013 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:47.925972 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" containerName="istio-proxy" Jul 14 11:51:47.926013 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:47.925995 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cd38f55-bde5-4361-8a98-a39f090e1817" containerName="istio-proxy" Jul 14 11:51:47.926013 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:47.926000 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cd38f55-bde5-4361-8a98-a39f090e1817" containerName="istio-proxy" Jul 14 11:51:47.926106 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:47.926067 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" containerName="istio-proxy" Jul 14 11:51:47.926106 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:47.926076 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" containerName="istio-proxy" Jul 14 11:51:47.930649 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:47.930609 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:47.934268 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:47.934250 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-67b298da-openshift-default-dockercfg-x9tkd\"" Jul 14 11:51:47.942113 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:47.942094 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8"] Jul 14 11:51:47.957178 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:47.957159 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:48.037953 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.037921 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-envoy\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.038126 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.037965 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0a7015d-088b-4aa0-92f1-be8236d0544d-istiod-ca-cert\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.038126 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.038037 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-workload-socket\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.038126 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.038091 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-credential-socket\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.038126 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.038112 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-token\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.038300 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.038136 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqz7j\" (UniqueName: \"kubernetes.io/projected/c0a7015d-088b-4aa0-92f1-be8236d0544d-kube-api-access-fqz7j\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.038300 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.038186 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-data\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.038300 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.038276 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-podinfo\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.038401 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.038341 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-workload-certs\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.138742 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.138715 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-podinfo\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.138883 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.138765 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-workload-certs\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.138883 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.138873 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-envoy\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.138996 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.138907 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0a7015d-088b-4aa0-92f1-be8236d0544d-istiod-ca-cert\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.138996 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.138960 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-workload-socket\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.139103 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.139003 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-credential-socket\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.139103 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.139028 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-token\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.139103 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.139052 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqz7j\" (UniqueName: \"kubernetes.io/projected/c0a7015d-088b-4aa0-92f1-be8236d0544d-kube-api-access-fqz7j\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.139103 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.139081 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-data\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.139103 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.139090 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-workload-certs\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.139348 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.139317 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-workload-socket\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.139521 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.139502 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-credential-socket\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.139592 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.139570 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-data\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.139676 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.139589 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0a7015d-088b-4aa0-92f1-be8236d0544d-istiod-ca-cert\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.141142 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.141121 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-envoy\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.141268 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.141250 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-podinfo\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.159229 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.159200 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqz7j\" (UniqueName: \"kubernetes.io/projected/c0a7015d-088b-4aa0-92f1-be8236d0544d-kube-api-access-fqz7j\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.162019 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.162000 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-token\") pod \"e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.240581 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.240528 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:48.317864 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.317833 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:48.377818 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.377792 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8"] Jul 14 11:51:48.378990 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:51:48.378965 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0a7015d_088b_4aa0_92f1_be8236d0544d.slice/crio-cbcdaef1bd021821193a1d7be993a103323bedd2c07de068b097e9efac3b3184 WatchSource:0}: Error finding container cbcdaef1bd021821193a1d7be993a103323bedd2c07de068b097e9efac3b3184: Status 404 returned error can't find the container with id cbcdaef1bd021821193a1d7be993a103323bedd2c07de068b097e9efac3b3184 Jul 14 11:51:48.381011 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.380979 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:51:48.381143 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.381037 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:51:48.381143 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.381061 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:51:48.614392 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.614354 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" event={"ID":"c0a7015d-088b-4aa0-92f1-be8236d0544d","Type":"ContainerStarted","Data":"465dfde73f7e87b2971ba6aabc14f7799992bdf3739d18479497db418eec311b"} Jul 14 11:51:48.614531 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.614407 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" event={"ID":"c0a7015d-088b-4aa0-92f1-be8236d0544d","Type":"ContainerStarted","Data":"cbcdaef1bd021821193a1d7be993a103323bedd2c07de068b097e9efac3b3184"} Jul 14 11:51:48.640677 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:48.640617 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" podStartSLOduration=1.640602208 podStartE2EDuration="1.640602208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:51:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:51:48.638834348 +0000 UTC m=+2002.799910814" watchObservedRunningTime="2026-07-14 11:51:48.640602208 +0000 UTC m=+2002.801678674" Jul 14 11:51:49.240940 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:49.240910 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:49.245916 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:49.245893 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:49.620131 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:49.620104 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:49.620862 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:49.620845 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:51:50.582814 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:50.582760 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:50.687939 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:50.687911 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:50.764973 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:50.764938 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:50.888670 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:50.888643 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:51.251448 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:51.251371 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:51.348477 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:51.348449 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:51:55.119127 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:51:55.119093 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:52:03.704828 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:03.704794 2571 generic.go:356] "Generic (PLEG): container finished" podID="c0a7015d-088b-4aa0-92f1-be8236d0544d" containerID="465dfde73f7e87b2971ba6aabc14f7799992bdf3739d18479497db418eec311b" exitCode=0 Jul 14 11:52:03.705289 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:03.704838 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" event={"ID":"c0a7015d-088b-4aa0-92f1-be8236d0544d","Type":"ContainerDied","Data":"465dfde73f7e87b2971ba6aabc14f7799992bdf3739d18479497db418eec311b"} Jul 14 11:52:03.705289 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:03.705208 2571 scope.go:117] "RemoveContainer" containerID="465dfde73f7e87b2971ba6aabc14f7799992bdf3739d18479497db418eec311b" Jul 14 11:52:03.706222 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:03.706193 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:52:03.706275 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:03.706262 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:52:03.706314 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:03.706296 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:52:04.586940 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:04.586907 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:52:04.620432 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:04.620381 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:52:04.687136 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:04.687053 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:52:04.712119 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:04.712082 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" event={"ID":"c0a7015d-088b-4aa0-92f1-be8236d0544d","Type":"ContainerStarted","Data":"b8b3d0913000dbc989bf9360859c76426f57934bed1b188565d3077c22dff98b"} Jul 14 11:52:05.240980 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:05.240937 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:52:05.242164 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:05.242138 2571 patch_prober.go:28] interesting pod/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 14 11:52:05.242269 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:05.242186 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" podUID="c0a7015d-088b-4aa0-92f1-be8236d0544d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 14 11:52:06.241126 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:06.241083 2571 patch_prober.go:28] interesting pod/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 14 11:52:06.241515 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:06.241140 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" podUID="c0a7015d-088b-4aa0-92f1-be8236d0544d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 14 11:52:07.241382 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:07.241343 2571 patch_prober.go:28] interesting pod/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 14 11:52:07.241795 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:07.241402 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" podUID="c0a7015d-088b-4aa0-92f1-be8236d0544d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 14 11:52:07.796204 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:07.796164 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:52:08.241396 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:08.241360 2571 patch_prober.go:28] interesting pod/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 14 11:52:08.241785 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:08.241411 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" podUID="c0a7015d-088b-4aa0-92f1-be8236d0544d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 14 11:52:09.244824 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:09.244791 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:52:09.245249 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:09.245072 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:52:09.245686 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:09.245668 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:52:12.042749 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:12.042716 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:52:12.143789 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:12.143753 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:52:42.741468 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:42.741433 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:52:54.305158 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:54.305083 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:52:58.704517 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:52:58.704476 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:53:03.784107 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:03.784073 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:53:26.493581 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:26.493553 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 11:53:26.501549 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:26.501529 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 11:53:26.567389 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:26.567363 2571 scope.go:117] "RemoveContainer" containerID="4644fe6c37489beedad142186065b0e2d5d1446085b39522ebdf4fc715a8e965" Jul 14 11:53:39.901749 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:39.901715 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:53:41.187263 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:41.187230 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:53:47.092335 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:47.092303 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:53:47.193013 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:47.192981 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:53:51.055681 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:51.055624 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8"] Jul 14 11:53:51.056076 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:51.055850 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" podUID="c0a7015d-088b-4aa0-92f1-be8236d0544d" containerName="istio-proxy" containerID="cri-o://b8b3d0913000dbc989bf9360859c76426f57934bed1b188565d3077c22dff98b" gracePeriod=30 Jul 14 11:53:51.076961 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:51.076935 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:53:51.315946 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:51.315869 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:53:52.384732 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:52.384700 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:53:52.481405 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:52.481371 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:53:52.588072 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:52.588041 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:53:53.352526 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:53.352486 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:53:53.445087 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:53.445050 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:53:53.544351 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:53.544325 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:53:54.247533 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:54.247497 2571 patch_prober.go:28] interesting pod/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 11:53:54.247719 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:54.247562 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" podUID="c0a7015d-088b-4aa0-92f1-be8236d0544d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 11:53:55.109030 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:55.109000 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:53:56.305853 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.305830 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:53:56.321083 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.321008 2571 generic.go:356] "Generic (PLEG): container finished" podID="c0a7015d-088b-4aa0-92f1-be8236d0544d" containerID="b8b3d0913000dbc989bf9360859c76426f57934bed1b188565d3077c22dff98b" exitCode=0 Jul 14 11:53:56.321083 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.321065 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" Jul 14 11:53:56.321083 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.321072 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" event={"ID":"c0a7015d-088b-4aa0-92f1-be8236d0544d","Type":"ContainerDied","Data":"b8b3d0913000dbc989bf9360859c76426f57934bed1b188565d3077c22dff98b"} Jul 14 11:53:56.321273 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.321162 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8" event={"ID":"c0a7015d-088b-4aa0-92f1-be8236d0544d","Type":"ContainerDied","Data":"cbcdaef1bd021821193a1d7be993a103323bedd2c07de068b097e9efac3b3184"} Jul 14 11:53:56.321273 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.321188 2571 scope.go:117] "RemoveContainer" containerID="b8b3d0913000dbc989bf9360859c76426f57934bed1b188565d3077c22dff98b" Jul 14 11:53:56.329655 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.329619 2571 scope.go:117] "RemoveContainer" containerID="465dfde73f7e87b2971ba6aabc14f7799992bdf3739d18479497db418eec311b" Jul 14 11:53:56.338712 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.338689 2571 scope.go:117] "RemoveContainer" containerID="b8b3d0913000dbc989bf9360859c76426f57934bed1b188565d3077c22dff98b" Jul 14 11:53:56.338981 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:53:56.338961 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8b3d0913000dbc989bf9360859c76426f57934bed1b188565d3077c22dff98b\": container with ID starting with b8b3d0913000dbc989bf9360859c76426f57934bed1b188565d3077c22dff98b not found: ID does not exist" containerID="b8b3d0913000dbc989bf9360859c76426f57934bed1b188565d3077c22dff98b" Jul 14 11:53:56.339047 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.338993 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8b3d0913000dbc989bf9360859c76426f57934bed1b188565d3077c22dff98b"} err="failed to get container status \"b8b3d0913000dbc989bf9360859c76426f57934bed1b188565d3077c22dff98b\": rpc error: code = NotFound desc = could not find container \"b8b3d0913000dbc989bf9360859c76426f57934bed1b188565d3077c22dff98b\": container with ID starting with b8b3d0913000dbc989bf9360859c76426f57934bed1b188565d3077c22dff98b not found: ID does not exist" Jul 14 11:53:56.339047 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.339018 2571 scope.go:117] "RemoveContainer" containerID="465dfde73f7e87b2971ba6aabc14f7799992bdf3739d18479497db418eec311b" Jul 14 11:53:56.339297 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:53:56.339273 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"465dfde73f7e87b2971ba6aabc14f7799992bdf3739d18479497db418eec311b\": container with ID starting with 465dfde73f7e87b2971ba6aabc14f7799992bdf3739d18479497db418eec311b not found: ID does not exist" containerID="465dfde73f7e87b2971ba6aabc14f7799992bdf3739d18479497db418eec311b" Jul 14 11:53:56.339341 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.339305 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"465dfde73f7e87b2971ba6aabc14f7799992bdf3739d18479497db418eec311b"} err="failed to get container status \"465dfde73f7e87b2971ba6aabc14f7799992bdf3739d18479497db418eec311b\": rpc error: code = NotFound desc = could not find container \"465dfde73f7e87b2971ba6aabc14f7799992bdf3739d18479497db418eec311b\": container with ID starting with 465dfde73f7e87b2971ba6aabc14f7799992bdf3739d18479497db418eec311b not found: ID does not exist" Jul 14 11:53:56.411859 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.411837 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-envoy\") pod \"c0a7015d-088b-4aa0-92f1-be8236d0544d\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " Jul 14 11:53:56.411949 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.411872 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0a7015d-088b-4aa0-92f1-be8236d0544d-istiod-ca-cert\") pod \"c0a7015d-088b-4aa0-92f1-be8236d0544d\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " Jul 14 11:53:56.411949 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.411912 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-workload-socket\") pod \"c0a7015d-088b-4aa0-92f1-be8236d0544d\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " Jul 14 11:53:56.411949 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.411941 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-token\") pod \"c0a7015d-088b-4aa0-92f1-be8236d0544d\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " Jul 14 11:53:56.412103 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.411976 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-credential-socket\") pod \"c0a7015d-088b-4aa0-92f1-be8236d0544d\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " Jul 14 11:53:56.412103 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.412064 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-workload-certs\") pod \"c0a7015d-088b-4aa0-92f1-be8236d0544d\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " Jul 14 11:53:56.412203 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.412134 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqz7j\" (UniqueName: \"kubernetes.io/projected/c0a7015d-088b-4aa0-92f1-be8236d0544d-kube-api-access-fqz7j\") pod \"c0a7015d-088b-4aa0-92f1-be8236d0544d\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " Jul 14 11:53:56.412203 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.412195 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-data\") pod \"c0a7015d-088b-4aa0-92f1-be8236d0544d\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " Jul 14 11:53:56.412308 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.412207 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c0a7015d-088b-4aa0-92f1-be8236d0544d" (UID: "c0a7015d-088b-4aa0-92f1-be8236d0544d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:53:56.412308 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.412228 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-podinfo\") pod \"c0a7015d-088b-4aa0-92f1-be8236d0544d\" (UID: \"c0a7015d-088b-4aa0-92f1-be8236d0544d\") " Jul 14 11:53:56.412442 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.412223 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c0a7015d-088b-4aa0-92f1-be8236d0544d" (UID: "c0a7015d-088b-4aa0-92f1-be8236d0544d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:53:56.412442 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.412332 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c0a7015d-088b-4aa0-92f1-be8236d0544d" (UID: "c0a7015d-088b-4aa0-92f1-be8236d0544d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:53:56.412442 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.412370 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0a7015d-088b-4aa0-92f1-be8236d0544d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c0a7015d-088b-4aa0-92f1-be8236d0544d" (UID: "c0a7015d-088b-4aa0-92f1-be8236d0544d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:53:56.412615 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.412602 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0a7015d-088b-4aa0-92f1-be8236d0544d-istiod-ca-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:53:56.412696 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.412616 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-workload-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:53:56.412696 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.412624 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-credential-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:53:56.412696 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.412654 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-workload-certs\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:53:56.413009 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.412988 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-data" (OuterVolumeSpecName: "istio-data") pod "c0a7015d-088b-4aa0-92f1-be8236d0544d" (UID: "c0a7015d-088b-4aa0-92f1-be8236d0544d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:53:56.414471 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.414449 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-token" (OuterVolumeSpecName: "istio-token") pod "c0a7015d-088b-4aa0-92f1-be8236d0544d" (UID: "c0a7015d-088b-4aa0-92f1-be8236d0544d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:53:56.414552 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.414481 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0a7015d-088b-4aa0-92f1-be8236d0544d-kube-api-access-fqz7j" (OuterVolumeSpecName: "kube-api-access-fqz7j") pod "c0a7015d-088b-4aa0-92f1-be8236d0544d" (UID: "c0a7015d-088b-4aa0-92f1-be8236d0544d"). InnerVolumeSpecName "kube-api-access-fqz7j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:53:56.414677 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.414656 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c0a7015d-088b-4aa0-92f1-be8236d0544d" (UID: "c0a7015d-088b-4aa0-92f1-be8236d0544d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:53:56.414728 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.414661 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c0a7015d-088b-4aa0-92f1-be8236d0544d" (UID: "c0a7015d-088b-4aa0-92f1-be8236d0544d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 11:53:56.513362 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.513338 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-envoy\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:53:56.513362 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.513358 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-token\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:53:56.513480 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.513367 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqz7j\" (UniqueName: \"kubernetes.io/projected/c0a7015d-088b-4aa0-92f1-be8236d0544d-kube-api-access-fqz7j\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:53:56.513480 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.513376 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-data\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:53:56.513480 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.513384 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0a7015d-088b-4aa0-92f1-be8236d0544d-istio-podinfo\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:53:56.658078 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.658054 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8"] Jul 14 11:53:56.664837 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:56.664818 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-67b298da-openshift-default-6d55f9c887-rgrf8"] Jul 14 11:53:58.403401 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:53:58.403362 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0a7015d-088b-4aa0-92f1-be8236d0544d" path="/var/lib/kubelet/pods/c0a7015d-088b-4aa0-92f1-be8236d0544d/volumes" Jul 14 11:54:00.881161 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:00.881128 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:00.981925 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:00.981896 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:01.622125 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:01.622095 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:07.710804 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:07.710756 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:07.819254 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:07.819214 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:07.910740 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:07.910707 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:18.528619 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:18.528527 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:18.912247 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:18.912218 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:39.975742 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:39.975708 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:45.967961 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:45.967927 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:46.065015 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.064971 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:46.425578 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.425545 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:46.672757 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.672726 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn"] Jul 14 11:54:46.673189 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.673169 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0a7015d-088b-4aa0-92f1-be8236d0544d" containerName="istio-proxy" Jul 14 11:54:46.673189 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.673191 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0a7015d-088b-4aa0-92f1-be8236d0544d" containerName="istio-proxy" Jul 14 11:54:46.673340 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.673205 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" containerName="istio-proxy" Jul 14 11:54:46.673340 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.673213 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="684ea306-42c2-4c9d-a1ba-0e9a17dcd5af" containerName="istio-proxy" Jul 14 11:54:46.673340 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.673227 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0a7015d-088b-4aa0-92f1-be8236d0544d" containerName="istio-proxy" Jul 14 11:54:46.673340 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.673237 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0a7015d-088b-4aa0-92f1-be8236d0544d" containerName="istio-proxy" Jul 14 11:54:46.673340 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.673328 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0a7015d-088b-4aa0-92f1-be8236d0544d" containerName="istio-proxy" Jul 14 11:54:46.673581 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.673350 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0a7015d-088b-4aa0-92f1-be8236d0544d" containerName="istio-proxy" Jul 14 11:54:46.676492 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.676435 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.679373 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.679356 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-8567a3-openshift-default-dockercfg-lbhfn\"" Jul 14 11:54:46.687865 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.687842 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-token\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.687980 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.687888 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/feb37a64-b04a-4867-90f4-552bcfd00e2a-istiod-ca-cert\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.687980 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.687947 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-workload-socket\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.687980 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.687977 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-workload-certs\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.688138 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.688066 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-credential-socket\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.688138 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.688128 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwrwb\" (UniqueName: \"kubernetes.io/projected/feb37a64-b04a-4867-90f4-552bcfd00e2a-kube-api-access-vwrwb\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.688223 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.688158 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-envoy\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.688223 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.688189 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-data\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.688223 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.688209 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-podinfo\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.689149 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.689125 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn"] Jul 14 11:54:46.789319 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.789284 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-podinfo\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.789495 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.789350 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-token\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.789495 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.789392 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/feb37a64-b04a-4867-90f4-552bcfd00e2a-istiod-ca-cert\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.789495 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.789432 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-workload-socket\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.789495 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.789461 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-workload-certs\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.789759 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.789531 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-credential-socket\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.789759 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.789572 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwrwb\" (UniqueName: \"kubernetes.io/projected/feb37a64-b04a-4867-90f4-552bcfd00e2a-kube-api-access-vwrwb\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.789759 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.789598 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-envoy\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.789759 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.789657 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-data\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.790058 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.790003 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-workload-certs\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.790058 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.790029 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-data\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.790248 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.790118 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-workload-socket\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.790311 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.790266 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-credential-socket\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.790530 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.790509 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/feb37a64-b04a-4867-90f4-552bcfd00e2a-istiod-ca-cert\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.792307 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.792287 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-envoy\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.792639 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.792606 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-podinfo\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.804819 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.804801 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-token\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.805168 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.805146 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwrwb\" (UniqueName: \"kubernetes.io/projected/feb37a64-b04a-4867-90f4-552bcfd00e2a-kube-api-access-vwrwb\") pod \"e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:46.988560 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:46.988491 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:47.127364 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:47.127339 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn"] Jul 14 11:54:47.128093 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:54:47.128068 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeb37a64_b04a_4867_90f4_552bcfd00e2a.slice/crio-1f167f49a80554d2abdc1c3d44617495d1f12c0d7f777186ff8bae27dd030aee WatchSource:0}: Error finding container 1f167f49a80554d2abdc1c3d44617495d1f12c0d7f777186ff8bae27dd030aee: Status 404 returned error can't find the container with id 1f167f49a80554d2abdc1c3d44617495d1f12c0d7f777186ff8bae27dd030aee Jul 14 11:54:47.130189 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:47.130158 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:54:47.130279 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:47.130216 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:54:47.130279 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:47.130265 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:54:47.590426 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:47.590385 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:47.591173 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:47.591137 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" event={"ID":"feb37a64-b04a-4867-90f4-552bcfd00e2a","Type":"ContainerStarted","Data":"ea15360756ee4bd306b94a8619b2657a62b1feefcef0eed9d380cba7bd420740"} Jul 14 11:54:47.591321 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:47.591251 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" event={"ID":"feb37a64-b04a-4867-90f4-552bcfd00e2a","Type":"ContainerStarted","Data":"1f167f49a80554d2abdc1c3d44617495d1f12c0d7f777186ff8bae27dd030aee"} Jul 14 11:54:47.615237 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:47.615184 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" podStartSLOduration=1.61517007 podStartE2EDuration="1.61517007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:54:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:54:47.613598158 +0000 UTC m=+2181.774674623" watchObservedRunningTime="2026-07-14 11:54:47.61517007 +0000 UTC m=+2181.776246537" Jul 14 11:54:47.988716 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:47.988685 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:48.994194 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:48.994164 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:49.603532 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:49.603492 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:49.604377 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:49.604358 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:54:49.628377 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:49.628346 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:49.720195 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:49.720165 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:49.800023 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:49.799990 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:50.035302 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:50.035215 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:50.403155 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:50.403127 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:50.516034 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:50.515999 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:54.177229 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:54.177199 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:54.585605 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:54.585528 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:54.957755 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:54.957722 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk"] Jul 14 11:54:54.961535 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:54.961515 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:54.965481 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:54.965463 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-738ce7-openshift-default-dockercfg-ztws8\"" Jul 14 11:54:55.031281 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.031258 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:55.053710 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.053681 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk"] Jul 14 11:54:55.066599 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.066570 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-data\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.066785 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.066654 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-podinfo\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.066785 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.066690 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-workload-certs\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.066785 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.066735 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-credential-socket\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.066997 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.066788 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-envoy\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.066997 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.066846 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istiod-ca-cert\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.066997 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.066877 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ggg7\" (UniqueName: \"kubernetes.io/projected/8488a46e-c8cd-40d1-8df4-b8335efb98ff-kube-api-access-2ggg7\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.066997 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.066901 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-token\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.066997 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.066933 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-workload-socket\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.167408 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.167379 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-credential-socket\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.167573 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.167414 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-envoy\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.167573 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.167451 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istiod-ca-cert\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.167573 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.167469 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ggg7\" (UniqueName: \"kubernetes.io/projected/8488a46e-c8cd-40d1-8df4-b8335efb98ff-kube-api-access-2ggg7\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.167573 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.167493 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-token\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.167573 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.167522 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-workload-socket\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.167877 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.167575 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-data\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.167877 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.167667 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-podinfo\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.167877 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.167702 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-workload-certs\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.168035 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.167901 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-credential-socket\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.168035 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.167978 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-workload-socket\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.168035 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.168019 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-data\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.168138 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.168124 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-workload-certs\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.168407 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.168390 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istiod-ca-cert\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.169779 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.169762 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-envoy\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.170066 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.170051 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-podinfo\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.177475 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.177453 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ggg7\" (UniqueName: \"kubernetes.io/projected/8488a46e-c8cd-40d1-8df4-b8335efb98ff-kube-api-access-2ggg7\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.177882 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.177725 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-token\") pod \"e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.272138 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.272063 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:55.369274 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.369235 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:55.413524 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.413496 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk"] Jul 14 11:54:55.414324 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:54:55.414300 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8488a46e_c8cd_40d1_8df4_b8335efb98ff.slice/crio-03631ed10b1cf6ecdae629edf2f84b8976a90a7a899fe7c9a1bf91a51d5a7c15 WatchSource:0}: Error finding container 03631ed10b1cf6ecdae629edf2f84b8976a90a7a899fe7c9a1bf91a51d5a7c15: Status 404 returned error can't find the container with id 03631ed10b1cf6ecdae629edf2f84b8976a90a7a899fe7c9a1bf91a51d5a7c15 Jul 14 11:54:55.416539 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.416501 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:54:55.416664 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.416584 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:54:55.416664 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.416650 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:54:55.637410 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.637367 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" event={"ID":"8488a46e-c8cd-40d1-8df4-b8335efb98ff","Type":"ContainerStarted","Data":"ee4cede1ed1cbb1411c20c0c69a046d171b6fc1279af6cc344424ecd16a52f8a"} Jul 14 11:54:55.637606 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.637430 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" event={"ID":"8488a46e-c8cd-40d1-8df4-b8335efb98ff","Type":"ContainerStarted","Data":"03631ed10b1cf6ecdae629edf2f84b8976a90a7a899fe7c9a1bf91a51d5a7c15"} Jul 14 11:54:55.660889 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:55.660840 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" podStartSLOduration=1.660823559 podStartE2EDuration="1.660823559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:54:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:54:55.659459683 +0000 UTC m=+2189.820536152" watchObservedRunningTime="2026-07-14 11:54:55.660823559 +0000 UTC m=+2189.821900026" Jul 14 11:54:56.272313 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:56.272276 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:56.276940 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:56.276919 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:56.647891 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:56.647853 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:56.649091 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:56.649052 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:54:57.742467 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:57.742344 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:57.835707 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:57.835673 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:57.932407 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:57.932380 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:58.162263 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:58.162230 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:58.556537 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:58.556454 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:54:58.656172 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:54:58.656143 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:55:02.186275 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:02.186231 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:55:05.645790 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:05.645743 2571 patch_prober.go:28] interesting pod/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 14 11:55:05.646186 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:05.645819 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" podUID="feb37a64-b04a-4867-90f4-552bcfd00e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 14 11:55:06.708918 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:06.708883 2571 generic.go:356] "Generic (PLEG): container finished" podID="feb37a64-b04a-4867-90f4-552bcfd00e2a" containerID="ea15360756ee4bd306b94a8619b2657a62b1feefcef0eed9d380cba7bd420740" exitCode=0 Jul 14 11:55:06.709355 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:06.708955 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" event={"ID":"feb37a64-b04a-4867-90f4-552bcfd00e2a","Type":"ContainerDied","Data":"ea15360756ee4bd306b94a8619b2657a62b1feefcef0eed9d380cba7bd420740"} Jul 14 11:55:06.709399 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:06.709370 2571 scope.go:117] "RemoveContainer" containerID="ea15360756ee4bd306b94a8619b2657a62b1feefcef0eed9d380cba7bd420740" Jul 14 11:55:06.710519 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:06.710492 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:55:06.710591 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:06.710578 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:55:06.710649 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:06.710620 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:55:07.716165 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:07.716130 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" event={"ID":"feb37a64-b04a-4867-90f4-552bcfd00e2a","Type":"ContainerStarted","Data":"8a9b0a2b3410f9e46fae3f4f41dd15aedab540686ccfa029a209635d49d1d5ed"} Jul 14 11:55:07.989610 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:07.989509 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:55:07.991382 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:07.989913 2571 patch_prober.go:28] interesting pod/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 14 11:55:07.991621 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:07.991594 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" podUID="feb37a64-b04a-4867-90f4-552bcfd00e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 14 11:55:08.989752 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:08.989695 2571 patch_prober.go:28] interesting pod/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 14 11:55:08.990250 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:08.989780 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" podUID="feb37a64-b04a-4867-90f4-552bcfd00e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 14 11:55:09.989924 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:09.989832 2571 patch_prober.go:28] interesting pod/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 14 11:55:09.990467 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:09.990435 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" podUID="feb37a64-b04a-4867-90f4-552bcfd00e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 14 11:55:10.989913 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:10.989868 2571 patch_prober.go:28] interesting pod/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 14 11:55:10.990397 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:10.989935 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" podUID="feb37a64-b04a-4867-90f4-552bcfd00e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 14 11:55:11.373945 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:11.373740 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:55:11.474372 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:11.474318 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:55:11.992138 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:11.992107 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:55:11.992614 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:11.992595 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:55:11.993350 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:11.993329 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:55:12.511365 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:12.510917 2571 patch_prober.go:28] interesting pod/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": read tcp 10.133.0.2:49758->10.133.0.53:15021: read: connection reset by peer" start-of-body= Jul 14 11:55:12.511365 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:12.510976 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" podUID="8488a46e-c8cd-40d1-8df4-b8335efb98ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": read tcp 10.133.0.2:49758->10.133.0.53:15021: read: connection reset by peer" Jul 14 11:55:12.745706 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:12.745600 2571 generic.go:356] "Generic (PLEG): container finished" podID="8488a46e-c8cd-40d1-8df4-b8335efb98ff" containerID="ee4cede1ed1cbb1411c20c0c69a046d171b6fc1279af6cc344424ecd16a52f8a" exitCode=0 Jul 14 11:55:12.745706 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:12.745665 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" event={"ID":"8488a46e-c8cd-40d1-8df4-b8335efb98ff","Type":"ContainerDied","Data":"ee4cede1ed1cbb1411c20c0c69a046d171b6fc1279af6cc344424ecd16a52f8a"} Jul 14 11:55:12.747543 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:12.747519 2571 scope.go:117] "RemoveContainer" containerID="ee4cede1ed1cbb1411c20c0c69a046d171b6fc1279af6cc344424ecd16a52f8a" Jul 14 11:55:12.753115 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:12.753079 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:55:12.753225 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:12.753147 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:55:12.753225 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:12.753184 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:55:13.752784 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:13.752743 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" event={"ID":"8488a46e-c8cd-40d1-8df4-b8335efb98ff","Type":"ContainerStarted","Data":"7d8b5939d40e00aeba7b6132ef65159b6e392d087f6417c9b8f192491cf0dd69"} Jul 14 11:55:14.273362 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:14.273321 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:55:14.273614 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:14.273451 2571 patch_prober.go:28] interesting pod/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 14 11:55:14.273614 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:14.273507 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" podUID="8488a46e-c8cd-40d1-8df4-b8335efb98ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 14 11:55:15.293170 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:15.293131 2571 patch_prober.go:28] interesting pod/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 14 11:55:15.293771 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:15.293189 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" podUID="8488a46e-c8cd-40d1-8df4-b8335efb98ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 14 11:55:15.293771 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:15.293581 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:55:16.273035 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:16.272998 2571 patch_prober.go:28] interesting pod/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 14 11:55:16.273363 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:16.273332 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" podUID="8488a46e-c8cd-40d1-8df4-b8335efb98ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 14 11:55:17.273202 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:17.273156 2571 patch_prober.go:28] interesting pod/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 14 11:55:17.273698 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:17.273224 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" podUID="8488a46e-c8cd-40d1-8df4-b8335efb98ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 14 11:55:18.275733 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:18.275705 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:55:18.276132 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:18.275956 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:55:18.276567 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:18.276551 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:55:19.499868 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:19.499835 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:55:19.599825 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:19.599791 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:55:32.613644 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:32.613606 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:55:51.128007 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:51.127929 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:55:51.941354 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:55:51.941318 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:56:11.285939 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:11.285906 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:56:12.192802 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:12.192772 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:56:12.854689 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:12.854655 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:56:12.954932 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:12.954907 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:56:22.917623 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:22.917587 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:56:27.431712 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:27.431674 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:56:33.231988 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:33.231954 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:56:33.332893 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:33.332850 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:56:38.134832 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:38.134799 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn"] Jul 14 11:56:38.135203 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:38.135015 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" podUID="feb37a64-b04a-4867-90f4-552bcfd00e2a" containerName="istio-proxy" containerID="cri-o://8a9b0a2b3410f9e46fae3f4f41dd15aedab540686ccfa029a209635d49d1d5ed" gracePeriod=30 Jul 14 11:56:38.161350 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:38.161318 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:56:38.414934 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:38.414862 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:56:39.297870 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:39.297836 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:56:39.397862 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:39.397829 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:56:41.994465 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:41.994432 2571 patch_prober.go:28] interesting pod/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 11:56:41.994833 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:41.994496 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" podUID="feb37a64-b04a-4867-90f4-552bcfd00e2a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 11:56:43.389743 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.389718 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:56:43.486273 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.486215 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-data\") pod \"feb37a64-b04a-4867-90f4-552bcfd00e2a\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " Jul 14 11:56:43.486273 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.486252 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-workload-certs\") pod \"feb37a64-b04a-4867-90f4-552bcfd00e2a\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " Jul 14 11:56:43.486273 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.486270 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-workload-socket\") pod \"feb37a64-b04a-4867-90f4-552bcfd00e2a\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " Jul 14 11:56:43.486481 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.486296 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-envoy\") pod \"feb37a64-b04a-4867-90f4-552bcfd00e2a\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " Jul 14 11:56:43.486481 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.486349 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-podinfo\") pod \"feb37a64-b04a-4867-90f4-552bcfd00e2a\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " Jul 14 11:56:43.486481 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.486371 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-credential-socket\") pod \"feb37a64-b04a-4867-90f4-552bcfd00e2a\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " Jul 14 11:56:43.486481 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.486408 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/feb37a64-b04a-4867-90f4-552bcfd00e2a-istiod-ca-cert\") pod \"feb37a64-b04a-4867-90f4-552bcfd00e2a\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " Jul 14 11:56:43.486481 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.486435 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-token\") pod \"feb37a64-b04a-4867-90f4-552bcfd00e2a\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " Jul 14 11:56:43.486773 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.486490 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwrwb\" (UniqueName: \"kubernetes.io/projected/feb37a64-b04a-4867-90f4-552bcfd00e2a-kube-api-access-vwrwb\") pod \"feb37a64-b04a-4867-90f4-552bcfd00e2a\" (UID: \"feb37a64-b04a-4867-90f4-552bcfd00e2a\") " Jul 14 11:56:43.486773 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.486513 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "feb37a64-b04a-4867-90f4-552bcfd00e2a" (UID: "feb37a64-b04a-4867-90f4-552bcfd00e2a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:56:43.486773 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.486546 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "feb37a64-b04a-4867-90f4-552bcfd00e2a" (UID: "feb37a64-b04a-4867-90f4-552bcfd00e2a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:56:43.486773 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.486668 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "feb37a64-b04a-4867-90f4-552bcfd00e2a" (UID: "feb37a64-b04a-4867-90f4-552bcfd00e2a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:56:43.486988 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.486780 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-credential-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:56:43.486988 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.486798 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-workload-certs\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:56:43.486988 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.486812 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-workload-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:56:43.487122 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.487017 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/feb37a64-b04a-4867-90f4-552bcfd00e2a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "feb37a64-b04a-4867-90f4-552bcfd00e2a" (UID: "feb37a64-b04a-4867-90f4-552bcfd00e2a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:56:43.487254 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.487229 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-data" (OuterVolumeSpecName: "istio-data") pod "feb37a64-b04a-4867-90f4-552bcfd00e2a" (UID: "feb37a64-b04a-4867-90f4-552bcfd00e2a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:56:43.488624 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.488604 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-token" (OuterVolumeSpecName: "istio-token") pod "feb37a64-b04a-4867-90f4-552bcfd00e2a" (UID: "feb37a64-b04a-4867-90f4-552bcfd00e2a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:56:43.488961 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.488930 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "feb37a64-b04a-4867-90f4-552bcfd00e2a" (UID: "feb37a64-b04a-4867-90f4-552bcfd00e2a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 11:56:43.489076 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.489022 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/feb37a64-b04a-4867-90f4-552bcfd00e2a-kube-api-access-vwrwb" (OuterVolumeSpecName: "kube-api-access-vwrwb") pod "feb37a64-b04a-4867-90f4-552bcfd00e2a" (UID: "feb37a64-b04a-4867-90f4-552bcfd00e2a"). InnerVolumeSpecName "kube-api-access-vwrwb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:56:43.489076 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.489029 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "feb37a64-b04a-4867-90f4-552bcfd00e2a" (UID: "feb37a64-b04a-4867-90f4-552bcfd00e2a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:56:43.587777 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.587750 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwrwb\" (UniqueName: \"kubernetes.io/projected/feb37a64-b04a-4867-90f4-552bcfd00e2a-kube-api-access-vwrwb\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:56:43.587777 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.587774 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-data\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:56:43.587902 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.587783 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-envoy\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:56:43.587902 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.587791 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-podinfo\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:56:43.587902 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.587798 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/feb37a64-b04a-4867-90f4-552bcfd00e2a-istiod-ca-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:56:43.587902 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:43.587805 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/feb37a64-b04a-4867-90f4-552bcfd00e2a-istio-token\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:56:44.234060 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:44.234025 2571 generic.go:356] "Generic (PLEG): container finished" podID="feb37a64-b04a-4867-90f4-552bcfd00e2a" containerID="8a9b0a2b3410f9e46fae3f4f41dd15aedab540686ccfa029a209635d49d1d5ed" exitCode=0 Jul 14 11:56:44.234225 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:44.234069 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" Jul 14 11:56:44.234225 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:44.234089 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" event={"ID":"feb37a64-b04a-4867-90f4-552bcfd00e2a","Type":"ContainerDied","Data":"8a9b0a2b3410f9e46fae3f4f41dd15aedab540686ccfa029a209635d49d1d5ed"} Jul 14 11:56:44.234225 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:44.234129 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn" event={"ID":"feb37a64-b04a-4867-90f4-552bcfd00e2a","Type":"ContainerDied","Data":"1f167f49a80554d2abdc1c3d44617495d1f12c0d7f777186ff8bae27dd030aee"} Jul 14 11:56:44.234225 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:44.234150 2571 scope.go:117] "RemoveContainer" containerID="8a9b0a2b3410f9e46fae3f4f41dd15aedab540686ccfa029a209635d49d1d5ed" Jul 14 11:56:44.243869 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:44.243849 2571 scope.go:117] "RemoveContainer" containerID="ea15360756ee4bd306b94a8619b2657a62b1feefcef0eed9d380cba7bd420740" Jul 14 11:56:44.251901 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:44.251640 2571 scope.go:117] "RemoveContainer" containerID="8a9b0a2b3410f9e46fae3f4f41dd15aedab540686ccfa029a209635d49d1d5ed" Jul 14 11:56:44.251901 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:56:44.251891 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a9b0a2b3410f9e46fae3f4f41dd15aedab540686ccfa029a209635d49d1d5ed\": container with ID starting with 8a9b0a2b3410f9e46fae3f4f41dd15aedab540686ccfa029a209635d49d1d5ed not found: ID does not exist" containerID="8a9b0a2b3410f9e46fae3f4f41dd15aedab540686ccfa029a209635d49d1d5ed" Jul 14 11:56:44.252005 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:44.251913 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a9b0a2b3410f9e46fae3f4f41dd15aedab540686ccfa029a209635d49d1d5ed"} err="failed to get container status \"8a9b0a2b3410f9e46fae3f4f41dd15aedab540686ccfa029a209635d49d1d5ed\": rpc error: code = NotFound desc = could not find container \"8a9b0a2b3410f9e46fae3f4f41dd15aedab540686ccfa029a209635d49d1d5ed\": container with ID starting with 8a9b0a2b3410f9e46fae3f4f41dd15aedab540686ccfa029a209635d49d1d5ed not found: ID does not exist" Jul 14 11:56:44.252005 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:44.251930 2571 scope.go:117] "RemoveContainer" containerID="ea15360756ee4bd306b94a8619b2657a62b1feefcef0eed9d380cba7bd420740" Jul 14 11:56:44.252173 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:56:44.252152 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea15360756ee4bd306b94a8619b2657a62b1feefcef0eed9d380cba7bd420740\": container with ID starting with ea15360756ee4bd306b94a8619b2657a62b1feefcef0eed9d380cba7bd420740 not found: ID does not exist" containerID="ea15360756ee4bd306b94a8619b2657a62b1feefcef0eed9d380cba7bd420740" Jul 14 11:56:44.252225 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:44.252182 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea15360756ee4bd306b94a8619b2657a62b1feefcef0eed9d380cba7bd420740"} err="failed to get container status \"ea15360756ee4bd306b94a8619b2657a62b1feefcef0eed9d380cba7bd420740\": rpc error: code = NotFound desc = could not find container \"ea15360756ee4bd306b94a8619b2657a62b1feefcef0eed9d380cba7bd420740\": container with ID starting with ea15360756ee4bd306b94a8619b2657a62b1feefcef0eed9d380cba7bd420740 not found: ID does not exist" Jul 14 11:56:44.257344 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:44.257323 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn"] Jul 14 11:56:44.265215 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:44.265194 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-8567a3-openshift-default-78fbcdbb69-bprrn"] Jul 14 11:56:44.403572 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:44.403536 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="feb37a64-b04a-4867-90f4-552bcfd00e2a" path="/var/lib/kubelet/pods/feb37a64-b04a-4867-90f4-552bcfd00e2a/volumes" Jul 14 11:56:49.226611 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:49.226573 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:56:53.647357 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:53.647325 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:56:59.664328 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:59.664294 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:56:59.765374 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:56:59.765341 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:04.338896 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:04.338865 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk"] Jul 14 11:57:04.339391 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:04.339091 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" podUID="8488a46e-c8cd-40d1-8df4-b8335efb98ff" containerName="istio-proxy" containerID="cri-o://7d8b5939d40e00aeba7b6132ef65159b6e392d087f6417c9b8f192491cf0dd69" gracePeriod=30 Jul 14 11:57:04.360791 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:04.360762 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:04.595336 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:04.595303 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:04.972079 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:04.971998 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:05.291393 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.291322 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2"] Jul 14 11:57:05.291710 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.291696 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="feb37a64-b04a-4867-90f4-552bcfd00e2a" containerName="istio-proxy" Jul 14 11:57:05.291761 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.291711 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="feb37a64-b04a-4867-90f4-552bcfd00e2a" containerName="istio-proxy" Jul 14 11:57:05.291761 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.291726 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="feb37a64-b04a-4867-90f4-552bcfd00e2a" containerName="istio-proxy" Jul 14 11:57:05.291761 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.291732 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="feb37a64-b04a-4867-90f4-552bcfd00e2a" containerName="istio-proxy" Jul 14 11:57:05.291858 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.291813 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="feb37a64-b04a-4867-90f4-552bcfd00e2a" containerName="istio-proxy" Jul 14 11:57:05.291858 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.291824 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="feb37a64-b04a-4867-90f4-552bcfd00e2a" containerName="istio-proxy" Jul 14 11:57:05.294711 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.294686 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.297566 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.297544 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-565f5c-openshift-default-dockercfg-ljpss\"" Jul 14 11:57:05.313971 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.313948 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2"] Jul 14 11:57:05.344754 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.344724 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:05.360892 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.360869 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-podinfo\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.360998 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.360903 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-workload-certs\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.360998 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.360935 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-data\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.360998 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.360953 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-token\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.360998 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.360976 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-envoy\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.361206 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.361001 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-credential-socket\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.361206 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.361032 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efdb252f-25e3-4a91-92bc-96506bf1019d-istiod-ca-cert\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.361206 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.361124 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-workload-socket\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.361206 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.361162 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rwrd\" (UniqueName: \"kubernetes.io/projected/efdb252f-25e3-4a91-92bc-96506bf1019d-kube-api-access-8rwrd\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.439719 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.439690 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:05.463682 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.462034 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-workload-socket\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.463682 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.462098 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rwrd\" (UniqueName: \"kubernetes.io/projected/efdb252f-25e3-4a91-92bc-96506bf1019d-kube-api-access-8rwrd\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.463682 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.462177 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-podinfo\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.463682 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.462215 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-workload-certs\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.463682 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.462273 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-data\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.463682 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.462310 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-token\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.463682 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.462341 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-envoy\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.463682 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.462394 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-credential-socket\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.463682 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.462458 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efdb252f-25e3-4a91-92bc-96506bf1019d-istiod-ca-cert\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.463682 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.462752 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-workload-socket\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.463682 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.463222 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efdb252f-25e3-4a91-92bc-96506bf1019d-istiod-ca-cert\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.463682 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.463508 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-workload-certs\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.464444 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.463763 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-data\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.464593 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.464571 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-credential-socket\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.466428 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.466359 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-envoy\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.467099 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.467055 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-podinfo\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.473710 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.473680 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-token\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.473969 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.473944 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rwrd\" (UniqueName: \"kubernetes.io/projected/efdb252f-25e3-4a91-92bc-96506bf1019d-kube-api-access-8rwrd\") pod \"e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.604777 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.604753 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:05.659262 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.659231 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:05.757982 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.757952 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2"] Jul 14 11:57:05.758817 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:57:05.758789 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefdb252f_25e3_4a91_92bc_96506bf1019d.slice/crio-84778dfb120a8210ce41fe91153045d1b0ee3fa5a1e0039ad1bda2882254f332 WatchSource:0}: Error finding container 84778dfb120a8210ce41fe91153045d1b0ee3fa5a1e0039ad1bda2882254f332: Status 404 returned error can't find the container with id 84778dfb120a8210ce41fe91153045d1b0ee3fa5a1e0039ad1bda2882254f332 Jul 14 11:57:05.761085 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.761049 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:57:05.761183 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.761118 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:57:05.761183 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.761146 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:57:05.764107 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:05.764087 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:06.354548 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:06.354505 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" event={"ID":"efdb252f-25e3-4a91-92bc-96506bf1019d","Type":"ContainerStarted","Data":"2f1942c4a2b963ac61f9c7ad52e92071e27659d0b88a487bb58e9fe8342fa807"} Jul 14 11:57:06.355028 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:06.354559 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" event={"ID":"efdb252f-25e3-4a91-92bc-96506bf1019d","Type":"ContainerStarted","Data":"84778dfb120a8210ce41fe91153045d1b0ee3fa5a1e0039ad1bda2882254f332"} Jul 14 11:57:06.377999 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:06.377953 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" podStartSLOduration=1.377937707 podStartE2EDuration="1.377937707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:57:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:57:06.375964157 +0000 UTC m=+2320.537040624" watchObservedRunningTime="2026-07-14 11:57:06.377937707 +0000 UTC m=+2320.539014173" Jul 14 11:57:06.605386 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:06.605308 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:06.609934 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:06.609911 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:07.359936 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:07.359893 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:07.360817 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:07.360795 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:08.305301 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:08.305272 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:08.415600 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:08.415567 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:08.510641 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:08.510595 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:08.633779 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:08.633751 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:09.026616 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.026540 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:09.126856 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.126822 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:09.595663 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.595622 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:57:09.701268 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.701198 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-envoy\") pod \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " Jul 14 11:57:09.701268 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.701246 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ggg7\" (UniqueName: \"kubernetes.io/projected/8488a46e-c8cd-40d1-8df4-b8335efb98ff-kube-api-access-2ggg7\") pod \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " Jul 14 11:57:09.701268 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.701276 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-workload-certs\") pod \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " Jul 14 11:57:09.701526 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.701297 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-podinfo\") pod \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " Jul 14 11:57:09.701526 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.701319 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-credential-socket\") pod \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " Jul 14 11:57:09.701526 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.701335 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istiod-ca-cert\") pod \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " Jul 14 11:57:09.701526 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.701350 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-data\") pod \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " Jul 14 11:57:09.701526 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.701398 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-workload-socket\") pod \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " Jul 14 11:57:09.701526 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.701434 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-token\") pod \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\" (UID: \"8488a46e-c8cd-40d1-8df4-b8335efb98ff\") " Jul 14 11:57:09.701872 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.701547 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8488a46e-c8cd-40d1-8df4-b8335efb98ff" (UID: "8488a46e-c8cd-40d1-8df4-b8335efb98ff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:57:09.701872 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.701567 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8488a46e-c8cd-40d1-8df4-b8335efb98ff" (UID: "8488a46e-c8cd-40d1-8df4-b8335efb98ff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:57:09.701872 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.701785 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8488a46e-c8cd-40d1-8df4-b8335efb98ff" (UID: "8488a46e-c8cd-40d1-8df4-b8335efb98ff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:57:09.701872 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.701838 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-workload-certs\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:57:09.701872 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.701859 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-credential-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:57:09.702135 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.702010 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8488a46e-c8cd-40d1-8df4-b8335efb98ff" (UID: "8488a46e-c8cd-40d1-8df4-b8335efb98ff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:57:09.702389 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.702345 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-data" (OuterVolumeSpecName: "istio-data") pod "8488a46e-c8cd-40d1-8df4-b8335efb98ff" (UID: "8488a46e-c8cd-40d1-8df4-b8335efb98ff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:57:09.703443 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.703415 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8488a46e-c8cd-40d1-8df4-b8335efb98ff" (UID: "8488a46e-c8cd-40d1-8df4-b8335efb98ff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:57:09.703664 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.703616 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-token" (OuterVolumeSpecName: "istio-token") pod "8488a46e-c8cd-40d1-8df4-b8335efb98ff" (UID: "8488a46e-c8cd-40d1-8df4-b8335efb98ff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:57:09.703930 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.703908 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8488a46e-c8cd-40d1-8df4-b8335efb98ff" (UID: "8488a46e-c8cd-40d1-8df4-b8335efb98ff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 11:57:09.703930 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.703920 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8488a46e-c8cd-40d1-8df4-b8335efb98ff-kube-api-access-2ggg7" (OuterVolumeSpecName: "kube-api-access-2ggg7") pod "8488a46e-c8cd-40d1-8df4-b8335efb98ff" (UID: "8488a46e-c8cd-40d1-8df4-b8335efb98ff"). InnerVolumeSpecName "kube-api-access-2ggg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:57:09.802852 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.802831 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-podinfo\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:57:09.802852 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.802850 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istiod-ca-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:57:09.802982 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.802860 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-data\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:57:09.802982 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.802868 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-workload-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:57:09.802982 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.802877 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-token\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:57:09.802982 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.802885 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8488a46e-c8cd-40d1-8df4-b8335efb98ff-istio-envoy\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:57:09.802982 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:09.802893 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ggg7\" (UniqueName: \"kubernetes.io/projected/8488a46e-c8cd-40d1-8df4-b8335efb98ff-kube-api-access-2ggg7\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:57:10.377177 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:10.377146 2571 generic.go:356] "Generic (PLEG): container finished" podID="8488a46e-c8cd-40d1-8df4-b8335efb98ff" containerID="7d8b5939d40e00aeba7b6132ef65159b6e392d087f6417c9b8f192491cf0dd69" exitCode=0 Jul 14 11:57:10.377365 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:10.377192 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" Jul 14 11:57:10.377365 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:10.377237 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" event={"ID":"8488a46e-c8cd-40d1-8df4-b8335efb98ff","Type":"ContainerDied","Data":"7d8b5939d40e00aeba7b6132ef65159b6e392d087f6417c9b8f192491cf0dd69"} Jul 14 11:57:10.377365 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:10.377298 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk" event={"ID":"8488a46e-c8cd-40d1-8df4-b8335efb98ff","Type":"ContainerDied","Data":"03631ed10b1cf6ecdae629edf2f84b8976a90a7a899fe7c9a1bf91a51d5a7c15"} Jul 14 11:57:10.377365 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:10.377332 2571 scope.go:117] "RemoveContainer" containerID="7d8b5939d40e00aeba7b6132ef65159b6e392d087f6417c9b8f192491cf0dd69" Jul 14 11:57:10.387287 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:10.387273 2571 scope.go:117] "RemoveContainer" containerID="ee4cede1ed1cbb1411c20c0c69a046d171b6fc1279af6cc344424ecd16a52f8a" Jul 14 11:57:10.395198 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:10.395179 2571 scope.go:117] "RemoveContainer" containerID="7d8b5939d40e00aeba7b6132ef65159b6e392d087f6417c9b8f192491cf0dd69" Jul 14 11:57:10.395451 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:57:10.395431 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d8b5939d40e00aeba7b6132ef65159b6e392d087f6417c9b8f192491cf0dd69\": container with ID starting with 7d8b5939d40e00aeba7b6132ef65159b6e392d087f6417c9b8f192491cf0dd69 not found: ID does not exist" containerID="7d8b5939d40e00aeba7b6132ef65159b6e392d087f6417c9b8f192491cf0dd69" Jul 14 11:57:10.395506 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:10.395459 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d8b5939d40e00aeba7b6132ef65159b6e392d087f6417c9b8f192491cf0dd69"} err="failed to get container status \"7d8b5939d40e00aeba7b6132ef65159b6e392d087f6417c9b8f192491cf0dd69\": rpc error: code = NotFound desc = could not find container \"7d8b5939d40e00aeba7b6132ef65159b6e392d087f6417c9b8f192491cf0dd69\": container with ID starting with 7d8b5939d40e00aeba7b6132ef65159b6e392d087f6417c9b8f192491cf0dd69 not found: ID does not exist" Jul 14 11:57:10.395506 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:10.395475 2571 scope.go:117] "RemoveContainer" containerID="ee4cede1ed1cbb1411c20c0c69a046d171b6fc1279af6cc344424ecd16a52f8a" Jul 14 11:57:10.395704 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:57:10.395688 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee4cede1ed1cbb1411c20c0c69a046d171b6fc1279af6cc344424ecd16a52f8a\": container with ID starting with ee4cede1ed1cbb1411c20c0c69a046d171b6fc1279af6cc344424ecd16a52f8a not found: ID does not exist" containerID="ee4cede1ed1cbb1411c20c0c69a046d171b6fc1279af6cc344424ecd16a52f8a" Jul 14 11:57:10.395752 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:10.395710 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee4cede1ed1cbb1411c20c0c69a046d171b6fc1279af6cc344424ecd16a52f8a"} err="failed to get container status \"ee4cede1ed1cbb1411c20c0c69a046d171b6fc1279af6cc344424ecd16a52f8a\": rpc error: code = NotFound desc = could not find container \"ee4cede1ed1cbb1411c20c0c69a046d171b6fc1279af6cc344424ecd16a52f8a\": container with ID starting with ee4cede1ed1cbb1411c20c0c69a046d171b6fc1279af6cc344424ecd16a52f8a not found: ID does not exist" Jul 14 11:57:10.404141 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:10.404123 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk"] Jul 14 11:57:10.408933 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:10.408914 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-738ce7-openshift-default-6f84d6bd6f-54vkk"] Jul 14 11:57:12.402895 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:12.402862 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8488a46e-c8cd-40d1-8df4-b8335efb98ff" path="/var/lib/kubelet/pods/8488a46e-c8cd-40d1-8df4-b8335efb98ff/volumes" Jul 14 11:57:12.777046 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:12.776966 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:13.228002 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.227969 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:13.525175 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.525105 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm"] Jul 14 11:57:13.525518 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.525465 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8488a46e-c8cd-40d1-8df4-b8335efb98ff" containerName="istio-proxy" Jul 14 11:57:13.525518 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.525479 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8488a46e-c8cd-40d1-8df4-b8335efb98ff" containerName="istio-proxy" Jul 14 11:57:13.525518 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.525499 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8488a46e-c8cd-40d1-8df4-b8335efb98ff" containerName="istio-proxy" Jul 14 11:57:13.525518 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.525505 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8488a46e-c8cd-40d1-8df4-b8335efb98ff" containerName="istio-proxy" Jul 14 11:57:13.525717 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.525583 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8488a46e-c8cd-40d1-8df4-b8335efb98ff" containerName="istio-proxy" Jul 14 11:57:13.530342 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.530321 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.533929 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.533909 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-290745-openshift-default-dockercfg-zmlv5\"" Jul 14 11:57:13.543021 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.542998 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm"] Jul 14 11:57:13.557481 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.557458 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:13.634512 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.634477 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-token\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.634811 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.634782 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-workload-certs\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.634999 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.634962 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istiod-ca-cert\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.635130 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.635016 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-data\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.635130 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.635057 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-credential-socket\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.635130 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.635097 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tszcd\" (UniqueName: \"kubernetes.io/projected/d41f4780-bdb4-4b0b-ac83-915c43425ab0-kube-api-access-tszcd\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.635295 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.635140 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-envoy\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.635295 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.635213 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-podinfo\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.635295 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.635248 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-workload-socket\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.736073 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.736043 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-data\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.736239 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.736081 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-credential-socket\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.736239 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.736109 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tszcd\" (UniqueName: \"kubernetes.io/projected/d41f4780-bdb4-4b0b-ac83-915c43425ab0-kube-api-access-tszcd\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.736371 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.736241 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-envoy\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.736371 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.736332 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-podinfo\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.736479 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.736381 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-workload-socket\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.736479 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.736391 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-data\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.736479 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.736449 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-credential-socket\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.736674 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.736496 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-token\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.736674 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.736541 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-workload-certs\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.736674 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.736579 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istiod-ca-cert\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.736833 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.736710 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-workload-socket\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.736875 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.736835 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-workload-certs\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.737303 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.737282 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istiod-ca-cert\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.738566 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.738542 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-envoy\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.738744 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.738726 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:13.739042 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.739023 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-podinfo\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.744908 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.744881 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-token\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.745012 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.744997 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tszcd\" (UniqueName: \"kubernetes.io/projected/d41f4780-bdb4-4b0b-ac83-915c43425ab0-kube-api-access-tszcd\") pod \"e2e-api-b-290745-openshift-default-64b6dcb955-826mm\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.840852 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.840779 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:13.984761 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.984734 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm"] Jul 14 11:57:13.985469 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:57:13.985446 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd41f4780_bdb4_4b0b_ac83_915c43425ab0.slice/crio-e54cefd1146a35d2a764816140698af212b075ca513fe6344d13fe6df071aae0 WatchSource:0}: Error finding container e54cefd1146a35d2a764816140698af212b075ca513fe6344d13fe6df071aae0: Status 404 returned error can't find the container with id e54cefd1146a35d2a764816140698af212b075ca513fe6344d13fe6df071aae0 Jul 14 11:57:13.987720 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.987692 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:57:13.987777 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.987756 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:57:13.987818 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:13.987786 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:57:14.403737 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:14.403703 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" event={"ID":"d41f4780-bdb4-4b0b-ac83-915c43425ab0","Type":"ContainerStarted","Data":"20f4ca790d3969e415e87122291db96df5e0f3add336735103d58404c1fa726e"} Jul 14 11:57:14.403906 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:14.403754 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" event={"ID":"d41f4780-bdb4-4b0b-ac83-915c43425ab0","Type":"ContainerStarted","Data":"e54cefd1146a35d2a764816140698af212b075ca513fe6344d13fe6df071aae0"} Jul 14 11:57:14.454146 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:14.454098 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" podStartSLOduration=1.454082694 podStartE2EDuration="1.454082694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:57:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:57:14.451112134 +0000 UTC m=+2328.612188602" watchObservedRunningTime="2026-07-14 11:57:14.454082694 +0000 UTC m=+2328.615159216" Jul 14 11:57:14.841910 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:14.841824 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:15.846539 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:15.846505 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:16.386498 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:16.386464 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:16.412787 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:16.412756 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:16.413706 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:16.413687 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:16.484114 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:16.484069 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:16.704464 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:16.704383 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:17.111322 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:17.111293 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:17.262540 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:17.262507 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:17.361903 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:17.361816 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:20.846429 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:20.846396 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:22.361581 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:22.361546 2571 patch_prober.go:28] interesting pod/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 14 11:57:22.362020 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:22.361599 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" podUID="efdb252f-25e3-4a91-92bc-96506bf1019d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 14 11:57:22.446446 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:22.446418 2571 generic.go:356] "Generic (PLEG): container finished" podID="efdb252f-25e3-4a91-92bc-96506bf1019d" containerID="2f1942c4a2b963ac61f9c7ad52e92071e27659d0b88a487bb58e9fe8342fa807" exitCode=0 Jul 14 11:57:22.446612 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:22.446492 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" event={"ID":"efdb252f-25e3-4a91-92bc-96506bf1019d","Type":"ContainerDied","Data":"2f1942c4a2b963ac61f9c7ad52e92071e27659d0b88a487bb58e9fe8342fa807"} Jul 14 11:57:22.446884 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:22.446870 2571 scope.go:117] "RemoveContainer" containerID="2f1942c4a2b963ac61f9c7ad52e92071e27659d0b88a487bb58e9fe8342fa807" Jul 14 11:57:22.447888 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:22.447864 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:57:22.447957 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:22.447927 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:57:22.447993 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:22.447961 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:57:23.454019 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:23.453982 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" event={"ID":"efdb252f-25e3-4a91-92bc-96506bf1019d","Type":"ContainerStarted","Data":"587727ed41133f01d81ca2c834239d31a0b03b8267aafd3da7a3a5a236b1322c"} Jul 14 11:57:23.605856 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:23.605809 2571 patch_prober.go:28] interesting pod/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 14 11:57:23.606068 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:23.605907 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" podUID="efdb252f-25e3-4a91-92bc-96506bf1019d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 14 11:57:23.606278 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:23.606259 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:24.605518 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:24.605468 2571 patch_prober.go:28] interesting pod/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 14 11:57:24.606150 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:24.606103 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" podUID="efdb252f-25e3-4a91-92bc-96506bf1019d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 14 11:57:25.605854 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:25.605810 2571 patch_prober.go:28] interesting pod/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 14 11:57:25.606241 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:25.605922 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" podUID="efdb252f-25e3-4a91-92bc-96506bf1019d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 14 11:57:26.605809 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:26.605775 2571 patch_prober.go:28] interesting pod/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 14 11:57:26.605968 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:26.605827 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" podUID="efdb252f-25e3-4a91-92bc-96506bf1019d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 14 11:57:26.697615 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:26.697586 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:27.609639 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:27.609597 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:27.610028 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:27.609881 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:27.610683 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:27.610664 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:57:28.486480 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:28.486449 2571 generic.go:356] "Generic (PLEG): container finished" podID="d41f4780-bdb4-4b0b-ac83-915c43425ab0" containerID="20f4ca790d3969e415e87122291db96df5e0f3add336735103d58404c1fa726e" exitCode=0 Jul 14 11:57:28.486674 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:28.486524 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" event={"ID":"d41f4780-bdb4-4b0b-ac83-915c43425ab0","Type":"ContainerDied","Data":"20f4ca790d3969e415e87122291db96df5e0f3add336735103d58404c1fa726e"} Jul 14 11:57:28.487037 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:28.487020 2571 scope.go:117] "RemoveContainer" containerID="20f4ca790d3969e415e87122291db96df5e0f3add336735103d58404c1fa726e" Jul 14 11:57:28.487973 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:28.487944 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:57:28.488067 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:28.488026 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:57:28.488134 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:28.488076 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:57:29.497934 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:29.497892 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" event={"ID":"d41f4780-bdb4-4b0b-ac83-915c43425ab0","Type":"ContainerStarted","Data":"ef43c6afa5416d3a4d71566b172dd1908117596112b576a26360e7f8c11601c8"} Jul 14 11:57:29.610953 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:29.610888 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:29.711500 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:29.711467 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:29.841280 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:29.841192 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:29.842285 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:29.842257 2571 patch_prober.go:28] interesting pod/e2e-api-b-290745-openshift-default-64b6dcb955-826mm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 14 11:57:29.842432 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:29.842322 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" podUID="d41f4780-bdb4-4b0b-ac83-915c43425ab0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 14 11:57:30.841977 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:30.841945 2571 patch_prober.go:28] interesting pod/e2e-api-b-290745-openshift-default-64b6dcb955-826mm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 14 11:57:30.842448 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:30.842005 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" podUID="d41f4780-bdb4-4b0b-ac83-915c43425ab0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 14 11:57:31.413445 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:31.413413 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:31.841842 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:31.841762 2571 patch_prober.go:28] interesting pod/e2e-api-b-290745-openshift-default-64b6dcb955-826mm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 14 11:57:31.841842 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:31.841812 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" podUID="d41f4780-bdb4-4b0b-ac83-915c43425ab0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 14 11:57:32.841105 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:32.841070 2571 patch_prober.go:28] interesting pod/e2e-api-b-290745-openshift-default-64b6dcb955-826mm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 14 11:57:32.841477 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:32.841121 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" podUID="d41f4780-bdb4-4b0b-ac83-915c43425ab0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 14 11:57:33.845166 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:33.845135 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:33.846177 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:33.846157 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:57:38.206082 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:38.206052 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:38.306837 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:38.306795 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:38.941844 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:38.941801 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:43.906171 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:43.906139 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:50.125953 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:50.125923 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:50.223705 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:50.223668 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:54.653987 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:54.653951 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2"] Jul 14 11:57:54.654449 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:54.654206 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" podUID="efdb252f-25e3-4a91-92bc-96506bf1019d" containerName="istio-proxy" containerID="cri-o://587727ed41133f01d81ca2c834239d31a0b03b8267aafd3da7a3a5a236b1322c" gracePeriod=30 Jul 14 11:57:54.675594 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:54.675564 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:54.870412 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:54.870373 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:57:57.611970 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:57.611935 2571 patch_prober.go:28] interesting pod/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 11:57:57.612330 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:57.611998 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" podUID="efdb252f-25e3-4a91-92bc-96506bf1019d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 11:57:59.912541 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:57:59.912511 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:58:00.066717 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.066655 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-token\") pod \"efdb252f-25e3-4a91-92bc-96506bf1019d\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " Jul 14 11:58:00.066717 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.066695 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-credential-socket\") pod \"efdb252f-25e3-4a91-92bc-96506bf1019d\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " Jul 14 11:58:00.066894 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.066726 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-envoy\") pod \"efdb252f-25e3-4a91-92bc-96506bf1019d\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " Jul 14 11:58:00.066894 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.066841 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efdb252f-25e3-4a91-92bc-96506bf1019d-istiod-ca-cert\") pod \"efdb252f-25e3-4a91-92bc-96506bf1019d\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " Jul 14 11:58:00.066894 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.066886 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-workload-certs\") pod \"efdb252f-25e3-4a91-92bc-96506bf1019d\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " Jul 14 11:58:00.067057 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.066915 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-podinfo\") pod \"efdb252f-25e3-4a91-92bc-96506bf1019d\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " Jul 14 11:58:00.067057 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.066948 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-data\") pod \"efdb252f-25e3-4a91-92bc-96506bf1019d\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " Jul 14 11:58:00.067057 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.067011 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-workload-socket\") pod \"efdb252f-25e3-4a91-92bc-96506bf1019d\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " Jul 14 11:58:00.067211 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.067056 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rwrd\" (UniqueName: \"kubernetes.io/projected/efdb252f-25e3-4a91-92bc-96506bf1019d-kube-api-access-8rwrd\") pod \"efdb252f-25e3-4a91-92bc-96506bf1019d\" (UID: \"efdb252f-25e3-4a91-92bc-96506bf1019d\") " Jul 14 11:58:00.067211 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.067044 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "efdb252f-25e3-4a91-92bc-96506bf1019d" (UID: "efdb252f-25e3-4a91-92bc-96506bf1019d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:58:00.067318 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.067204 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "efdb252f-25e3-4a91-92bc-96506bf1019d" (UID: "efdb252f-25e3-4a91-92bc-96506bf1019d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:58:00.067318 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.067285 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "efdb252f-25e3-4a91-92bc-96506bf1019d" (UID: "efdb252f-25e3-4a91-92bc-96506bf1019d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:58:00.067414 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.067390 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efdb252f-25e3-4a91-92bc-96506bf1019d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "efdb252f-25e3-4a91-92bc-96506bf1019d" (UID: "efdb252f-25e3-4a91-92bc-96506bf1019d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:58:00.067460 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.067403 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-workload-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:58:00.067460 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.067441 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-credential-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:58:00.067641 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.067459 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-workload-certs\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:58:00.067852 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.067828 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-data" (OuterVolumeSpecName: "istio-data") pod "efdb252f-25e3-4a91-92bc-96506bf1019d" (UID: "efdb252f-25e3-4a91-92bc-96506bf1019d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:58:00.068848 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.068822 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "efdb252f-25e3-4a91-92bc-96506bf1019d" (UID: "efdb252f-25e3-4a91-92bc-96506bf1019d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:58:00.069189 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.069167 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efdb252f-25e3-4a91-92bc-96506bf1019d-kube-api-access-8rwrd" (OuterVolumeSpecName: "kube-api-access-8rwrd") pod "efdb252f-25e3-4a91-92bc-96506bf1019d" (UID: "efdb252f-25e3-4a91-92bc-96506bf1019d"). InnerVolumeSpecName "kube-api-access-8rwrd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:58:00.069385 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.069367 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-token" (OuterVolumeSpecName: "istio-token") pod "efdb252f-25e3-4a91-92bc-96506bf1019d" (UID: "efdb252f-25e3-4a91-92bc-96506bf1019d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:58:00.069432 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.069371 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "efdb252f-25e3-4a91-92bc-96506bf1019d" (UID: "efdb252f-25e3-4a91-92bc-96506bf1019d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 11:58:00.168095 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.168063 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efdb252f-25e3-4a91-92bc-96506bf1019d-istiod-ca-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:58:00.168095 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.168092 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-podinfo\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:58:00.168095 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.168100 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-data\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:58:00.168252 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.168110 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rwrd\" (UniqueName: \"kubernetes.io/projected/efdb252f-25e3-4a91-92bc-96506bf1019d-kube-api-access-8rwrd\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:58:00.168252 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.168118 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-token\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:58:00.168252 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.168126 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efdb252f-25e3-4a91-92bc-96506bf1019d-istio-envoy\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:58:00.676315 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.676246 2571 generic.go:356] "Generic (PLEG): container finished" podID="efdb252f-25e3-4a91-92bc-96506bf1019d" containerID="587727ed41133f01d81ca2c834239d31a0b03b8267aafd3da7a3a5a236b1322c" exitCode=0 Jul 14 11:58:00.676315 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.676293 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" Jul 14 11:58:00.676467 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.676327 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" event={"ID":"efdb252f-25e3-4a91-92bc-96506bf1019d","Type":"ContainerDied","Data":"587727ed41133f01d81ca2c834239d31a0b03b8267aafd3da7a3a5a236b1322c"} Jul 14 11:58:00.676467 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.676380 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2" event={"ID":"efdb252f-25e3-4a91-92bc-96506bf1019d","Type":"ContainerDied","Data":"84778dfb120a8210ce41fe91153045d1b0ee3fa5a1e0039ad1bda2882254f332"} Jul 14 11:58:00.676467 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.676410 2571 scope.go:117] "RemoveContainer" containerID="587727ed41133f01d81ca2c834239d31a0b03b8267aafd3da7a3a5a236b1322c" Jul 14 11:58:00.685467 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.685448 2571 scope.go:117] "RemoveContainer" containerID="2f1942c4a2b963ac61f9c7ad52e92071e27659d0b88a487bb58e9fe8342fa807" Jul 14 11:58:00.693501 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.693483 2571 scope.go:117] "RemoveContainer" containerID="587727ed41133f01d81ca2c834239d31a0b03b8267aafd3da7a3a5a236b1322c" Jul 14 11:58:00.693744 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:58:00.693726 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"587727ed41133f01d81ca2c834239d31a0b03b8267aafd3da7a3a5a236b1322c\": container with ID starting with 587727ed41133f01d81ca2c834239d31a0b03b8267aafd3da7a3a5a236b1322c not found: ID does not exist" containerID="587727ed41133f01d81ca2c834239d31a0b03b8267aafd3da7a3a5a236b1322c" Jul 14 11:58:00.693802 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.693758 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"587727ed41133f01d81ca2c834239d31a0b03b8267aafd3da7a3a5a236b1322c"} err="failed to get container status \"587727ed41133f01d81ca2c834239d31a0b03b8267aafd3da7a3a5a236b1322c\": rpc error: code = NotFound desc = could not find container \"587727ed41133f01d81ca2c834239d31a0b03b8267aafd3da7a3a5a236b1322c\": container with ID starting with 587727ed41133f01d81ca2c834239d31a0b03b8267aafd3da7a3a5a236b1322c not found: ID does not exist" Jul 14 11:58:00.693802 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.693776 2571 scope.go:117] "RemoveContainer" containerID="2f1942c4a2b963ac61f9c7ad52e92071e27659d0b88a487bb58e9fe8342fa807" Jul 14 11:58:00.694004 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:58:00.693990 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f1942c4a2b963ac61f9c7ad52e92071e27659d0b88a487bb58e9fe8342fa807\": container with ID starting with 2f1942c4a2b963ac61f9c7ad52e92071e27659d0b88a487bb58e9fe8342fa807 not found: ID does not exist" containerID="2f1942c4a2b963ac61f9c7ad52e92071e27659d0b88a487bb58e9fe8342fa807" Jul 14 11:58:00.694045 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.694008 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f1942c4a2b963ac61f9c7ad52e92071e27659d0b88a487bb58e9fe8342fa807"} err="failed to get container status \"2f1942c4a2b963ac61f9c7ad52e92071e27659d0b88a487bb58e9fe8342fa807\": rpc error: code = NotFound desc = could not find container \"2f1942c4a2b963ac61f9c7ad52e92071e27659d0b88a487bb58e9fe8342fa807\": container with ID starting with 2f1942c4a2b963ac61f9c7ad52e92071e27659d0b88a487bb58e9fe8342fa807 not found: ID does not exist" Jul 14 11:58:00.698351 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.698331 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2"] Jul 14 11:58:00.703057 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:00.703038 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-565f5c-openshift-default-c455bcc87-mtgj2"] Jul 14 11:58:01.674303 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:01.674263 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:02.402929 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:02.402895 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efdb252f-25e3-4a91-92bc-96506bf1019d" path="/var/lib/kubelet/pods/efdb252f-25e3-4a91-92bc-96506bf1019d/volumes" Jul 14 11:58:05.497207 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:05.497168 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:10.081392 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:10.081360 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:16.187857 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:16.187822 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:16.265064 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:16.265023 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:20.774548 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:20.774515 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm"] Jul 14 11:58:20.775051 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:20.774796 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" podUID="d41f4780-bdb4-4b0b-ac83-915c43425ab0" containerName="istio-proxy" containerID="cri-o://ef43c6afa5416d3a4d71566b172dd1908117596112b576a26360e7f8c11601c8" gracePeriod=30 Jul 14 11:58:20.795975 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:20.795946 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:21.024340 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.024304 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:21.368167 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.368128 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:21.674420 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.674346 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m"] Jul 14 11:58:21.674778 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.674762 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efdb252f-25e3-4a91-92bc-96506bf1019d" containerName="istio-proxy" Jul 14 11:58:21.674867 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.674780 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="efdb252f-25e3-4a91-92bc-96506bf1019d" containerName="istio-proxy" Jul 14 11:58:21.674867 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.674796 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efdb252f-25e3-4a91-92bc-96506bf1019d" containerName="istio-proxy" Jul 14 11:58:21.674867 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.674802 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="efdb252f-25e3-4a91-92bc-96506bf1019d" containerName="istio-proxy" Jul 14 11:58:21.674998 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.674880 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="efdb252f-25e3-4a91-92bc-96506bf1019d" containerName="istio-proxy" Jul 14 11:58:21.674998 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.674890 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8488a46e-c8cd-40d1-8df4-b8335efb98ff" containerName="istio-proxy" Jul 14 11:58:21.674998 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.674898 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="efdb252f-25e3-4a91-92bc-96506bf1019d" containerName="istio-proxy" Jul 14 11:58:21.677889 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.677873 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.681124 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.681099 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-1cc796-openshift-default-dockercfg-77llg\"" Jul 14 11:58:21.698573 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.698554 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m"] Jul 14 11:58:21.716818 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.716795 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:21.745975 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.745955 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-credential-socket\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.746079 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.745996 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-istio-data\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.746079 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.746023 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5216258f-a445-4253-81bd-10154d5ecf28-istio-podinfo\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.746158 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.746079 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-workload-socket\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.746158 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.746115 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-workload-certs\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.746158 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.746137 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5216258f-a445-4253-81bd-10154d5ecf28-istiod-ca-cert\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.746273 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.746170 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t6mw\" (UniqueName: \"kubernetes.io/projected/5216258f-a445-4253-81bd-10154d5ecf28-kube-api-access-2t6mw\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.746273 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.746209 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5216258f-a445-4253-81bd-10154d5ecf28-istio-token\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.746273 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.746261 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-istio-envoy\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.847524 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.847488 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t6mw\" (UniqueName: \"kubernetes.io/projected/5216258f-a445-4253-81bd-10154d5ecf28-kube-api-access-2t6mw\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.848024 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.847535 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5216258f-a445-4253-81bd-10154d5ecf28-istio-token\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.848024 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.847585 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-istio-envoy\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.848024 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.847647 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-credential-socket\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.848024 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.847716 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-istio-data\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.848024 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.847761 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5216258f-a445-4253-81bd-10154d5ecf28-istio-podinfo\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.848024 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.847804 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-workload-socket\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.848024 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.847840 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-workload-certs\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.848024 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.847872 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5216258f-a445-4253-81bd-10154d5ecf28-istiod-ca-cert\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.848411 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.848141 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-credential-socket\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.848411 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.848200 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-istio-data\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.848411 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.848360 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-workload-socket\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.848531 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.848486 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-workload-certs\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.848876 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.848847 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5216258f-a445-4253-81bd-10154d5ecf28-istiod-ca-cert\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.850327 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.850303 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-istio-envoy\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.850495 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.850478 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5216258f-a445-4253-81bd-10154d5ecf28-istio-podinfo\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.856118 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.856089 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5216258f-a445-4253-81bd-10154d5ecf28-istio-token\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.856118 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.856106 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t6mw\" (UniqueName: \"kubernetes.io/projected/5216258f-a445-4253-81bd-10154d5ecf28-kube-api-access-2t6mw\") pod \"e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:21.863700 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.863670 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:21.988175 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:21.988102 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:22.129074 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:22.129049 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m"] Jul 14 11:58:22.130175 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:58:22.130150 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5216258f_a445_4253_81bd_10154d5ecf28.slice/crio-008be0e713fc27664d0f19af782c9af8bfa020f1a5dd3bbbf764458c15e1bdd6 WatchSource:0}: Error finding container 008be0e713fc27664d0f19af782c9af8bfa020f1a5dd3bbbf764458c15e1bdd6: Status 404 returned error can't find the container with id 008be0e713fc27664d0f19af782c9af8bfa020f1a5dd3bbbf764458c15e1bdd6 Jul 14 11:58:22.132775 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:22.132743 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:58:22.132863 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:22.132806 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:58:22.132863 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:22.132834 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:58:22.794477 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:22.794435 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" event={"ID":"5216258f-a445-4253-81bd-10154d5ecf28","Type":"ContainerStarted","Data":"7dc7077ed25be8ac143ae4c069da3f298f9310904cdae81dd73114bdebfc9ef2"} Jul 14 11:58:22.794669 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:22.794493 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" event={"ID":"5216258f-a445-4253-81bd-10154d5ecf28","Type":"ContainerStarted","Data":"008be0e713fc27664d0f19af782c9af8bfa020f1a5dd3bbbf764458c15e1bdd6"} Jul 14 11:58:22.818184 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:22.818139 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" podStartSLOduration=1.8181248939999999 podStartE2EDuration="1.818124894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:58:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:58:22.814913474 +0000 UTC m=+2396.975989941" watchObservedRunningTime="2026-07-14 11:58:22.818124894 +0000 UTC m=+2396.979201360" Jul 14 11:58:22.988459 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:22.988426 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:23.993009 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:23.992981 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:24.805047 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:24.805013 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:24.806320 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:24.806293 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:24.809458 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:24.809437 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:24.901294 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:24.901202 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:25.003941 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:25.003912 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:25.124136 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:25.124090 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:25.547268 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:25.547179 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:25.649137 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:25.649081 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:26.029197 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.029172 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:58:26.091446 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.091419 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-workload-certs\") pod \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " Jul 14 11:58:26.091567 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.091463 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-envoy\") pod \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " Jul 14 11:58:26.091567 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.091490 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-data\") pod \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " Jul 14 11:58:26.091567 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.091530 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-credential-socket\") pod \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " Jul 14 11:58:26.091567 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.091560 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tszcd\" (UniqueName: \"kubernetes.io/projected/d41f4780-bdb4-4b0b-ac83-915c43425ab0-kube-api-access-tszcd\") pod \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " Jul 14 11:58:26.091779 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.091651 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-workload-socket\") pod \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " Jul 14 11:58:26.091779 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.091693 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istiod-ca-cert\") pod \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " Jul 14 11:58:26.091779 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.091713 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d41f4780-bdb4-4b0b-ac83-915c43425ab0" (UID: "d41f4780-bdb4-4b0b-ac83-915c43425ab0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:58:26.091935 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.091852 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-token\") pod \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " Jul 14 11:58:26.091935 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.091900 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-podinfo\") pod \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\" (UID: \"d41f4780-bdb4-4b0b-ac83-915c43425ab0\") " Jul 14 11:58:26.092097 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.091927 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d41f4780-bdb4-4b0b-ac83-915c43425ab0" (UID: "d41f4780-bdb4-4b0b-ac83-915c43425ab0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:58:26.092097 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.092024 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d41f4780-bdb4-4b0b-ac83-915c43425ab0" (UID: "d41f4780-bdb4-4b0b-ac83-915c43425ab0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 11:58:26.092492 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.092469 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-workload-certs\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:58:26.092595 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.092498 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-credential-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:58:26.092595 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.092513 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istiod-ca-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:58:26.092718 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.092597 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d41f4780-bdb4-4b0b-ac83-915c43425ab0" (UID: "d41f4780-bdb4-4b0b-ac83-915c43425ab0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:58:26.093125 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.093103 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-data" (OuterVolumeSpecName: "istio-data") pod "d41f4780-bdb4-4b0b-ac83-915c43425ab0" (UID: "d41f4780-bdb4-4b0b-ac83-915c43425ab0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:58:26.094129 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.094110 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d41f4780-bdb4-4b0b-ac83-915c43425ab0" (UID: "d41f4780-bdb4-4b0b-ac83-915c43425ab0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 11:58:26.094329 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.094303 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d41f4780-bdb4-4b0b-ac83-915c43425ab0-kube-api-access-tszcd" (OuterVolumeSpecName: "kube-api-access-tszcd") pod "d41f4780-bdb4-4b0b-ac83-915c43425ab0" (UID: "d41f4780-bdb4-4b0b-ac83-915c43425ab0"). InnerVolumeSpecName "kube-api-access-tszcd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:58:26.094329 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.094317 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-token" (OuterVolumeSpecName: "istio-token") pod "d41f4780-bdb4-4b0b-ac83-915c43425ab0" (UID: "d41f4780-bdb4-4b0b-ac83-915c43425ab0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 11:58:26.094443 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.094313 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d41f4780-bdb4-4b0b-ac83-915c43425ab0" (UID: "d41f4780-bdb4-4b0b-ac83-915c43425ab0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 11:58:26.193622 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.193562 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-workload-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:58:26.193622 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.193586 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-token\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:58:26.193622 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.193600 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-podinfo\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:58:26.193622 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.193611 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-envoy\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:58:26.193622 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.193623 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d41f4780-bdb4-4b0b-ac83-915c43425ab0-istio-data\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:58:26.193869 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.193654 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tszcd\" (UniqueName: \"kubernetes.io/projected/d41f4780-bdb4-4b0b-ac83-915c43425ab0-kube-api-access-tszcd\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 11:58:26.532051 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.531992 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 11:58:26.541437 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.541419 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 11:58:26.816720 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.816616 2571 generic.go:356] "Generic (PLEG): container finished" podID="d41f4780-bdb4-4b0b-ac83-915c43425ab0" containerID="ef43c6afa5416d3a4d71566b172dd1908117596112b576a26360e7f8c11601c8" exitCode=0 Jul 14 11:58:26.816720 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.816685 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" Jul 14 11:58:26.816720 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.816701 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" event={"ID":"d41f4780-bdb4-4b0b-ac83-915c43425ab0","Type":"ContainerDied","Data":"ef43c6afa5416d3a4d71566b172dd1908117596112b576a26360e7f8c11601c8"} Jul 14 11:58:26.816957 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.816760 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm" event={"ID":"d41f4780-bdb4-4b0b-ac83-915c43425ab0","Type":"ContainerDied","Data":"e54cefd1146a35d2a764816140698af212b075ca513fe6344d13fe6df071aae0"} Jul 14 11:58:26.816957 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.816793 2571 scope.go:117] "RemoveContainer" containerID="ef43c6afa5416d3a4d71566b172dd1908117596112b576a26360e7f8c11601c8" Jul 14 11:58:26.825584 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.825567 2571 scope.go:117] "RemoveContainer" containerID="20f4ca790d3969e415e87122291db96df5e0f3add336735103d58404c1fa726e" Jul 14 11:58:26.833039 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.833022 2571 scope.go:117] "RemoveContainer" containerID="ef43c6afa5416d3a4d71566b172dd1908117596112b576a26360e7f8c11601c8" Jul 14 11:58:26.833288 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:58:26.833270 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef43c6afa5416d3a4d71566b172dd1908117596112b576a26360e7f8c11601c8\": container with ID starting with ef43c6afa5416d3a4d71566b172dd1908117596112b576a26360e7f8c11601c8 not found: ID does not exist" containerID="ef43c6afa5416d3a4d71566b172dd1908117596112b576a26360e7f8c11601c8" Jul 14 11:58:26.833349 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.833295 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef43c6afa5416d3a4d71566b172dd1908117596112b576a26360e7f8c11601c8"} err="failed to get container status \"ef43c6afa5416d3a4d71566b172dd1908117596112b576a26360e7f8c11601c8\": rpc error: code = NotFound desc = could not find container \"ef43c6afa5416d3a4d71566b172dd1908117596112b576a26360e7f8c11601c8\": container with ID starting with ef43c6afa5416d3a4d71566b172dd1908117596112b576a26360e7f8c11601c8 not found: ID does not exist" Jul 14 11:58:26.833349 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.833310 2571 scope.go:117] "RemoveContainer" containerID="20f4ca790d3969e415e87122291db96df5e0f3add336735103d58404c1fa726e" Jul 14 11:58:26.833537 ip-10-0-133-87 kubenswrapper[2571]: E0714 11:58:26.833519 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20f4ca790d3969e415e87122291db96df5e0f3add336735103d58404c1fa726e\": container with ID starting with 20f4ca790d3969e415e87122291db96df5e0f3add336735103d58404c1fa726e not found: ID does not exist" containerID="20f4ca790d3969e415e87122291db96df5e0f3add336735103d58404c1fa726e" Jul 14 11:58:26.833575 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.833545 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20f4ca790d3969e415e87122291db96df5e0f3add336735103d58404c1fa726e"} err="failed to get container status \"20f4ca790d3969e415e87122291db96df5e0f3add336735103d58404c1fa726e\": rpc error: code = NotFound desc = could not find container \"20f4ca790d3969e415e87122291db96df5e0f3add336735103d58404c1fa726e\": container with ID starting with 20f4ca790d3969e415e87122291db96df5e0f3add336735103d58404c1fa726e not found: ID does not exist" Jul 14 11:58:26.842352 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.842332 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm"] Jul 14 11:58:26.846536 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:26.846513 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-290745-openshift-default-64b6dcb955-826mm"] Jul 14 11:58:28.403279 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:28.403248 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d41f4780-bdb4-4b0b-ac83-915c43425ab0" path="/var/lib/kubelet/pods/d41f4780-bdb4-4b0b-ac83-915c43425ab0/volumes" Jul 14 11:58:29.331535 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:29.331498 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:29.643534 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:29.643486 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:29.935235 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:29.935152 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j"] Jul 14 11:58:29.935549 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:29.935536 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d41f4780-bdb4-4b0b-ac83-915c43425ab0" containerName="istio-proxy" Jul 14 11:58:29.935600 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:29.935550 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d41f4780-bdb4-4b0b-ac83-915c43425ab0" containerName="istio-proxy" Jul 14 11:58:29.935600 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:29.935565 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d41f4780-bdb4-4b0b-ac83-915c43425ab0" containerName="istio-proxy" Jul 14 11:58:29.935600 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:29.935571 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d41f4780-bdb4-4b0b-ac83-915c43425ab0" containerName="istio-proxy" Jul 14 11:58:29.935722 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:29.935667 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d41f4780-bdb4-4b0b-ac83-915c43425ab0" containerName="istio-proxy" Jul 14 11:58:29.935722 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:29.935679 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d41f4780-bdb4-4b0b-ac83-915c43425ab0" containerName="istio-proxy" Jul 14 11:58:29.940240 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:29.940218 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:29.943892 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:29.943873 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-0508a3-openshift-default-dockercfg-jg456\"" Jul 14 11:58:29.956237 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:29.956214 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j"] Jul 14 11:58:29.981355 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:29.981329 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:30.022072 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.022043 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2af99f0f-534f-40f0-8ade-3f3cfe195634-istiod-ca-cert\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.022302 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.022281 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-podinfo\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.022450 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.022434 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh45c\" (UniqueName: \"kubernetes.io/projected/2af99f0f-534f-40f0-8ade-3f3cfe195634-kube-api-access-mh45c\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.022601 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.022585 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-credential-socket\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.022830 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.022802 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-data\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.022951 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.022867 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-token\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.022951 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.022900 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-workload-socket\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.022951 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.022946 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-envoy\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.023126 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.022984 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-workload-certs\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.123772 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.123738 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-token\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.123974 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.123781 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-workload-socket\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.123974 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.123921 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-envoy\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.124114 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.123990 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-workload-certs\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.124114 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.124045 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2af99f0f-534f-40f0-8ade-3f3cfe195634-istiod-ca-cert\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.124114 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.124099 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-podinfo\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.124298 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.124125 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh45c\" (UniqueName: \"kubernetes.io/projected/2af99f0f-534f-40f0-8ade-3f3cfe195634-kube-api-access-mh45c\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.124298 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.124166 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-credential-socket\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.124298 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.124265 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-data\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.124466 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.124413 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-workload-certs\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.124971 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.124166 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-workload-socket\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.124971 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.124877 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2af99f0f-534f-40f0-8ade-3f3cfe195634-istiod-ca-cert\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.124971 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.124913 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-credential-socket\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.125224 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.125001 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-data\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.127192 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.127152 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-envoy\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.127361 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.127345 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-podinfo\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.133384 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.133358 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-token\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.133677 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.133658 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh45c\" (UniqueName: \"kubernetes.io/projected/2af99f0f-534f-40f0-8ade-3f3cfe195634-kube-api-access-mh45c\") pod \"e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.250303 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.250229 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:30.303142 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.303003 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:30.391076 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.391044 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j"] Jul 14 11:58:30.392135 ip-10-0-133-87 kubenswrapper[2571]: W0714 11:58:30.392103 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2af99f0f_534f_40f0_8ade_3f3cfe195634.slice/crio-852834b8b62599316f2969f9ca0da63db7619bb65eeffe5cfbca7ed1679ec5c0 WatchSource:0}: Error finding container 852834b8b62599316f2969f9ca0da63db7619bb65eeffe5cfbca7ed1679ec5c0: Status 404 returned error can't find the container with id 852834b8b62599316f2969f9ca0da63db7619bb65eeffe5cfbca7ed1679ec5c0 Jul 14 11:58:30.394612 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.394573 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:58:30.394750 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.394659 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:58:30.394750 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.394689 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:58:30.841432 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.841398 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" event={"ID":"2af99f0f-534f-40f0-8ade-3f3cfe195634","Type":"ContainerStarted","Data":"eda508e16026cae776c45015827ef065e924ee927294373825e69028ee9af915"} Jul 14 11:58:30.841955 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.841461 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" event={"ID":"2af99f0f-534f-40f0-8ade-3f3cfe195634","Type":"ContainerStarted","Data":"852834b8b62599316f2969f9ca0da63db7619bb65eeffe5cfbca7ed1679ec5c0"} Jul 14 11:58:30.864557 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:30.864475 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" podStartSLOduration=1.864459321 podStartE2EDuration="1.864459321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 11:58:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 11:58:30.862796901 +0000 UTC m=+2405.023873405" watchObservedRunningTime="2026-07-14 11:58:30.864459321 +0000 UTC m=+2405.025535789" Jul 14 11:58:31.250710 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:31.250675 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:31.252367 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:31.252340 2571 patch_prober.go:28] interesting pod/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 14 11:58:31.252499 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:31.252390 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" podUID="2af99f0f-534f-40f0-8ade-3f3cfe195634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 14 11:58:32.254274 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:32.254245 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:32.812913 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:32.812881 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:32.853066 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:32.853032 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:32.854068 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:32.854050 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:32.920115 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:32.920043 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:33.010945 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:33.010909 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:33.151802 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:33.151770 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:33.621599 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:33.621564 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:33.719034 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:33.719001 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:37.347377 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:37.347344 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:37.884692 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:37.884609 2571 generic.go:356] "Generic (PLEG): container finished" podID="5216258f-a445-4253-81bd-10154d5ecf28" containerID="7dc7077ed25be8ac143ae4c069da3f298f9310904cdae81dd73114bdebfc9ef2" exitCode=0 Jul 14 11:58:37.884826 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:37.884687 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" event={"ID":"5216258f-a445-4253-81bd-10154d5ecf28","Type":"ContainerDied","Data":"7dc7077ed25be8ac143ae4c069da3f298f9310904cdae81dd73114bdebfc9ef2"} Jul 14 11:58:37.885039 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:37.885025 2571 scope.go:117] "RemoveContainer" containerID="7dc7077ed25be8ac143ae4c069da3f298f9310904cdae81dd73114bdebfc9ef2" Jul 14 11:58:37.886078 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:37.886054 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:58:37.886136 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:37.886121 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:58:37.886171 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:37.886152 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:58:38.895696 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:38.895649 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" event={"ID":"5216258f-a445-4253-81bd-10154d5ecf28","Type":"ContainerStarted","Data":"f7302bf5baa57f4969206f37d19e0d12107b5966f6dd7fc3cf80ea8572bc821d"} Jul 14 11:58:38.990087 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:38.989152 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:38.990497 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:38.990471 2571 patch_prober.go:28] interesting pod/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 14 11:58:38.990719 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:38.990679 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" podUID="5216258f-a445-4253-81bd-10154d5ecf28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 14 11:58:39.805513 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:39.805479 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:39.988815 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:39.988778 2571 patch_prober.go:28] interesting pod/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 14 11:58:39.989399 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:39.989365 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" podUID="5216258f-a445-4253-81bd-10154d5ecf28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 14 11:58:40.988874 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:40.988832 2571 patch_prober.go:28] interesting pod/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 14 11:58:40.989375 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:40.988897 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" podUID="5216258f-a445-4253-81bd-10154d5ecf28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 14 11:58:41.988713 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:41.988677 2571 patch_prober.go:28] interesting pod/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 14 11:58:41.988984 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:41.988955 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" podUID="5216258f-a445-4253-81bd-10154d5ecf28" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 14 11:58:42.994264 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:42.994234 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:42.995159 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:42.995138 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 11:58:46.482179 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:46.482141 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:46.583834 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:46.583801 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:47.854755 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:47.854687 2571 patch_prober.go:28] interesting pod/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 14 11:58:47.855134 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:47.854748 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" podUID="2af99f0f-534f-40f0-8ade-3f3cfe195634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 14 11:58:47.959265 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:47.959235 2571 generic.go:356] "Generic (PLEG): container finished" podID="2af99f0f-534f-40f0-8ade-3f3cfe195634" containerID="eda508e16026cae776c45015827ef065e924ee927294373825e69028ee9af915" exitCode=0 Jul 14 11:58:47.959408 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:47.959306 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" event={"ID":"2af99f0f-534f-40f0-8ade-3f3cfe195634","Type":"ContainerDied","Data":"eda508e16026cae776c45015827ef065e924ee927294373825e69028ee9af915"} Jul 14 11:58:47.959675 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:47.959657 2571 scope.go:117] "RemoveContainer" containerID="eda508e16026cae776c45015827ef065e924ee927294373825e69028ee9af915" Jul 14 11:58:47.960607 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:47.960582 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:58:47.960692 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:47.960665 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:58:47.960732 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:47.960707 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 11:58:48.966255 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:48.966217 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" event={"ID":"2af99f0f-534f-40f0-8ade-3f3cfe195634","Type":"ContainerStarted","Data":"ce4d0df78954f1d01234288d137928ce4d5f7d6f197df2e9cc26a70c1eb730b1"} Jul 14 11:58:49.250562 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:49.250483 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:49.250733 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:49.250711 2571 patch_prober.go:28] interesting pod/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 14 11:58:49.250780 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:49.250761 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" podUID="2af99f0f-534f-40f0-8ade-3f3cfe195634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 14 11:58:50.250921 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:50.250874 2571 patch_prober.go:28] interesting pod/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 14 11:58:50.251302 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:50.250930 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" podUID="2af99f0f-534f-40f0-8ade-3f3cfe195634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 14 11:58:51.250589 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:51.250554 2571 patch_prober.go:28] interesting pod/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 14 11:58:51.250782 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:51.250607 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" podUID="2af99f0f-534f-40f0-8ade-3f3cfe195634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 14 11:58:52.250759 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:52.250722 2571 patch_prober.go:28] interesting pod/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 14 11:58:52.251129 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:52.250774 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" podUID="2af99f0f-534f-40f0-8ade-3f3cfe195634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 14 11:58:53.255542 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:53.255513 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:53.256082 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:53.255846 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:53.256465 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:53.256448 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 11:58:53.605308 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:53.605276 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:55.824554 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:55.824521 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:58:55.929709 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:58:55.929678 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:59:25.211455 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:59:25.211419 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 11:59:41.644213 ip-10-0-133-87 kubenswrapper[2571]: I0714 11:59:41.644179 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:01:15.974532 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:01:15.974498 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:01:18.928994 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:01:18.928964 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:01:33.806954 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:01:33.806920 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:01:34.913179 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:01:34.913143 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:01:52.562824 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:01:52.562746 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:02:06.026353 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:02:06.026312 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:02:06.583806 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:02:06.583774 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:02:06.681094 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:02:06.681068 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:02:21.562447 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:02:21.562411 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:02:31.090264 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:02:31.090228 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:02:42.197522 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:02:42.197490 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:02:43.182204 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:02:43.182165 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:03:00.288012 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:03:00.287981 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:03:26.568097 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:03:26.568024 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 12:03:26.577364 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:03:26.577342 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cglt_5baf7735-3ad0-40de-b92f-1b77fc61c592/ovn-acl-logging/0.log" Jul 14 12:03:27.499895 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:03:27.499856 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:03:47.400286 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:03:47.400256 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:03:47.502174 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:03:47.502143 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:03:56.606688 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:03:56.606625 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:03:56.706927 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:03:56.706895 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:04:23.135675 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:04:23.135617 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:04:24.194250 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:04:24.194213 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:04:43.496194 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:04:43.496099 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:04:43.596464 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:04:43.596428 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:04:53.637075 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:04:53.636988 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:04:58.113951 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:04:58.113912 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:05:04.442561 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:04.442526 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:05:04.542862 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:04.542825 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:05:08.793335 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:08.793297 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m"] Jul 14 12:05:08.793737 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:08.793602 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" podUID="5216258f-a445-4253-81bd-10154d5ecf28" containerName="istio-proxy" containerID="cri-o://f7302bf5baa57f4969206f37d19e0d12107b5966f6dd7fc3cf80ea8572bc821d" gracePeriod=30 Jul 14 12:05:08.812466 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:08.812442 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:05:09.035026 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:09.034991 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:05:09.727215 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:09.727181 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:05:09.826053 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:09.826022 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:05:12.997331 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:12.997300 2571 patch_prober.go:28] interesting pod/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 12:05:12.997716 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:12.997363 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" podUID="5216258f-a445-4253-81bd-10154d5ecf28" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 12:05:14.038723 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.038692 2571 generic.go:356] "Generic (PLEG): container finished" podID="5216258f-a445-4253-81bd-10154d5ecf28" containerID="f7302bf5baa57f4969206f37d19e0d12107b5966f6dd7fc3cf80ea8572bc821d" exitCode=0 Jul 14 12:05:14.039041 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.038742 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" event={"ID":"5216258f-a445-4253-81bd-10154d5ecf28","Type":"ContainerDied","Data":"f7302bf5baa57f4969206f37d19e0d12107b5966f6dd7fc3cf80ea8572bc821d"} Jul 14 12:05:14.039041 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.038810 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" event={"ID":"5216258f-a445-4253-81bd-10154d5ecf28","Type":"ContainerDied","Data":"008be0e713fc27664d0f19af782c9af8bfa020f1a5dd3bbbf764458c15e1bdd6"} Jul 14 12:05:14.039041 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.038827 2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="008be0e713fc27664d0f19af782c9af8bfa020f1a5dd3bbbf764458c15e1bdd6" Jul 14 12:05:14.039041 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.038838 2571 scope.go:117] "RemoveContainer" containerID="7dc7077ed25be8ac143ae4c069da3f298f9310904cdae81dd73114bdebfc9ef2" Jul 14 12:05:14.045520 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.045501 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 12:05:14.090803 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.090747 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-workload-certs\") pod \"5216258f-a445-4253-81bd-10154d5ecf28\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " Jul 14 12:05:14.090803 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.090793 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-workload-socket\") pod \"5216258f-a445-4253-81bd-10154d5ecf28\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " Jul 14 12:05:14.090958 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.090846 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2t6mw\" (UniqueName: \"kubernetes.io/projected/5216258f-a445-4253-81bd-10154d5ecf28-kube-api-access-2t6mw\") pod \"5216258f-a445-4253-81bd-10154d5ecf28\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " Jul 14 12:05:14.090958 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.090872 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-istio-data\") pod \"5216258f-a445-4253-81bd-10154d5ecf28\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " Jul 14 12:05:14.090958 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.090910 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5216258f-a445-4253-81bd-10154d5ecf28-istio-podinfo\") pod \"5216258f-a445-4253-81bd-10154d5ecf28\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " Jul 14 12:05:14.091117 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.090966 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-credential-socket\") pod \"5216258f-a445-4253-81bd-10154d5ecf28\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " Jul 14 12:05:14.091117 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.091008 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5216258f-a445-4253-81bd-10154d5ecf28-istio-token\") pod \"5216258f-a445-4253-81bd-10154d5ecf28\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " Jul 14 12:05:14.091117 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.091008 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5216258f-a445-4253-81bd-10154d5ecf28" (UID: "5216258f-a445-4253-81bd-10154d5ecf28"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 12:05:14.091117 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.091048 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5216258f-a445-4253-81bd-10154d5ecf28-istiod-ca-cert\") pod \"5216258f-a445-4253-81bd-10154d5ecf28\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " Jul 14 12:05:14.091117 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.091072 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-istio-envoy\") pod \"5216258f-a445-4253-81bd-10154d5ecf28\" (UID: \"5216258f-a445-4253-81bd-10154d5ecf28\") " Jul 14 12:05:14.091117 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.091099 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5216258f-a445-4253-81bd-10154d5ecf28" (UID: "5216258f-a445-4253-81bd-10154d5ecf28"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 12:05:14.091427 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.091399 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-workload-certs\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 12:05:14.091427 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.091405 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5216258f-a445-4253-81bd-10154d5ecf28-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5216258f-a445-4253-81bd-10154d5ecf28" (UID: "5216258f-a445-4253-81bd-10154d5ecf28"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 12:05:14.091427 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.091417 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-workload-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 12:05:14.091573 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.091533 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5216258f-a445-4253-81bd-10154d5ecf28" (UID: "5216258f-a445-4253-81bd-10154d5ecf28"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 12:05:14.091889 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.091835 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-istio-data" (OuterVolumeSpecName: "istio-data") pod "5216258f-a445-4253-81bd-10154d5ecf28" (UID: "5216258f-a445-4253-81bd-10154d5ecf28"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 12:05:14.093004 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.092982 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5216258f-a445-4253-81bd-10154d5ecf28-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5216258f-a445-4253-81bd-10154d5ecf28" (UID: "5216258f-a445-4253-81bd-10154d5ecf28"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 12:05:14.093662 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.093622 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5216258f-a445-4253-81bd-10154d5ecf28-istio-token" (OuterVolumeSpecName: "istio-token") pod "5216258f-a445-4253-81bd-10154d5ecf28" (UID: "5216258f-a445-4253-81bd-10154d5ecf28"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 12:05:14.093754 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.093704 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5216258f-a445-4253-81bd-10154d5ecf28-kube-api-access-2t6mw" (OuterVolumeSpecName: "kube-api-access-2t6mw") pod "5216258f-a445-4253-81bd-10154d5ecf28" (UID: "5216258f-a445-4253-81bd-10154d5ecf28"). InnerVolumeSpecName "kube-api-access-2t6mw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 12:05:14.093754 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.093723 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5216258f-a445-4253-81bd-10154d5ecf28" (UID: "5216258f-a445-4253-81bd-10154d5ecf28"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 12:05:14.192291 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.192260 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2t6mw\" (UniqueName: \"kubernetes.io/projected/5216258f-a445-4253-81bd-10154d5ecf28-kube-api-access-2t6mw\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 12:05:14.192291 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.192287 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-istio-data\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 12:05:14.192291 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.192296 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5216258f-a445-4253-81bd-10154d5ecf28-istio-podinfo\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 12:05:14.192556 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.192303 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-credential-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 12:05:14.192556 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.192312 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5216258f-a445-4253-81bd-10154d5ecf28-istio-token\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 12:05:14.192556 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.192333 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5216258f-a445-4253-81bd-10154d5ecf28-istiod-ca-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 12:05:14.192556 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:14.192341 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5216258f-a445-4253-81bd-10154d5ecf28-istio-envoy\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 12:05:15.045298 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:15.045265 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m" Jul 14 12:05:15.082752 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:15.082719 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m"] Jul 14 12:05:15.087404 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:15.087381 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-1cc796-openshift-default-5c4b6c49cf-v454m"] Jul 14 12:05:16.403613 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:16.403579 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5216258f-a445-4253-81bd-10154d5ecf28" path="/var/lib/kubelet/pods/5216258f-a445-4253-81bd-10154d5ecf28/volumes" Jul 14 12:05:19.851965 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:19.851922 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:05:24.257259 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:24.257214 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:05:26.698641 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:26.698597 2571 scope.go:117] "RemoveContainer" containerID="f7302bf5baa57f4969206f37d19e0d12107b5966f6dd7fc3cf80ea8572bc821d" Jul 14 12:05:30.057868 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:30.057829 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:05:30.158433 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:30.158397 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:05:34.943661 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:34.943597 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j"] Jul 14 12:05:34.944119 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:34.943822 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" podUID="2af99f0f-534f-40f0-8ade-3f3cfe195634" containerName="istio-proxy" containerID="cri-o://ce4d0df78954f1d01234288d137928ce4d5f7d6f197df2e9cc26a70c1eb730b1" gracePeriod=30 Jul 14 12:05:34.958343 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:34.958315 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:05:35.247058 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:35.246981 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:05:38.258241 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:38.258204 2571 patch_prober.go:28] interesting pod/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 12:05:38.258599 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:38.258270 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" podUID="2af99f0f-534f-40f0-8ade-3f3cfe195634" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 12:05:40.181740 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.181706 2571 generic.go:356] "Generic (PLEG): container finished" podID="2af99f0f-534f-40f0-8ade-3f3cfe195634" containerID="ce4d0df78954f1d01234288d137928ce4d5f7d6f197df2e9cc26a70c1eb730b1" exitCode=0 Jul 14 12:05:40.182063 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.181770 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" event={"ID":"2af99f0f-534f-40f0-8ade-3f3cfe195634","Type":"ContainerDied","Data":"ce4d0df78954f1d01234288d137928ce4d5f7d6f197df2e9cc26a70c1eb730b1"} Jul 14 12:05:40.182063 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.181828 2571 scope.go:117] "RemoveContainer" containerID="eda508e16026cae776c45015827ef065e924ee927294373825e69028ee9af915" Jul 14 12:05:40.202063 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.202045 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 12:05:40.317666 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.317577 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-podinfo\") pod \"2af99f0f-534f-40f0-8ade-3f3cfe195634\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " Jul 14 12:05:40.317666 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.317611 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-credential-socket\") pod \"2af99f0f-534f-40f0-8ade-3f3cfe195634\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " Jul 14 12:05:40.317666 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.317660 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-token\") pod \"2af99f0f-534f-40f0-8ade-3f3cfe195634\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " Jul 14 12:05:40.317875 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.317743 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-envoy\") pod \"2af99f0f-534f-40f0-8ade-3f3cfe195634\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " Jul 14 12:05:40.317875 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.317773 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2af99f0f-534f-40f0-8ade-3f3cfe195634-istiod-ca-cert\") pod \"2af99f0f-534f-40f0-8ade-3f3cfe195634\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " Jul 14 12:05:40.317875 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.317797 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-data\") pod \"2af99f0f-534f-40f0-8ade-3f3cfe195634\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " Jul 14 12:05:40.317875 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.317835 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mh45c\" (UniqueName: \"kubernetes.io/projected/2af99f0f-534f-40f0-8ade-3f3cfe195634-kube-api-access-mh45c\") pod \"2af99f0f-534f-40f0-8ade-3f3cfe195634\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " Jul 14 12:05:40.318080 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.317939 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-workload-certs\") pod \"2af99f0f-534f-40f0-8ade-3f3cfe195634\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " Jul 14 12:05:40.318080 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.317967 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-workload-socket\") pod \"2af99f0f-534f-40f0-8ade-3f3cfe195634\" (UID: \"2af99f0f-534f-40f0-8ade-3f3cfe195634\") " Jul 14 12:05:40.318080 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.317971 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2af99f0f-534f-40f0-8ade-3f3cfe195634" (UID: "2af99f0f-534f-40f0-8ade-3f3cfe195634"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 12:05:40.318264 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.318170 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2af99f0f-534f-40f0-8ade-3f3cfe195634-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2af99f0f-534f-40f0-8ade-3f3cfe195634" (UID: "2af99f0f-534f-40f0-8ade-3f3cfe195634"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 12:05:40.318361 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.318239 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2af99f0f-534f-40f0-8ade-3f3cfe195634" (UID: "2af99f0f-534f-40f0-8ade-3f3cfe195634"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 12:05:40.318433 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.318343 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2af99f0f-534f-40f0-8ade-3f3cfe195634-istiod-ca-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 12:05:40.318485 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.318428 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-credential-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 12:05:40.318485 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.318451 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2af99f0f-534f-40f0-8ade-3f3cfe195634" (UID: "2af99f0f-534f-40f0-8ade-3f3cfe195634"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 12:05:40.318806 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.318781 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-data" (OuterVolumeSpecName: "istio-data") pod "2af99f0f-534f-40f0-8ade-3f3cfe195634" (UID: "2af99f0f-534f-40f0-8ade-3f3cfe195634"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 12:05:40.319883 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.319860 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2af99f0f-534f-40f0-8ade-3f3cfe195634" (UID: "2af99f0f-534f-40f0-8ade-3f3cfe195634"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 12:05:40.320135 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.320118 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2af99f0f-534f-40f0-8ade-3f3cfe195634-kube-api-access-mh45c" (OuterVolumeSpecName: "kube-api-access-mh45c") pod "2af99f0f-534f-40f0-8ade-3f3cfe195634" (UID: "2af99f0f-534f-40f0-8ade-3f3cfe195634"). InnerVolumeSpecName "kube-api-access-mh45c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 12:05:40.320395 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.320375 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2af99f0f-534f-40f0-8ade-3f3cfe195634" (UID: "2af99f0f-534f-40f0-8ade-3f3cfe195634"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 12:05:40.320467 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.320389 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-token" (OuterVolumeSpecName: "istio-token") pod "2af99f0f-534f-40f0-8ade-3f3cfe195634" (UID: "2af99f0f-534f-40f0-8ade-3f3cfe195634"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 12:05:40.418994 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.418973 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-workload-certs\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 12:05:40.418994 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.418997 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-workload-socket\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 12:05:40.419120 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.419006 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-podinfo\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 12:05:40.419120 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.419015 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-token\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 12:05:40.419120 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.419023 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-envoy\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 12:05:40.419120 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.419029 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2af99f0f-534f-40f0-8ade-3f3cfe195634-istio-data\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 12:05:40.419120 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.419037 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mh45c\" (UniqueName: \"kubernetes.io/projected/2af99f0f-534f-40f0-8ade-3f3cfe195634-kube-api-access-mh45c\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 14 12:05:40.717529 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:40.717504 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5f86cb7fff-gcb99_29adf428-76cd-41cd-9ee6-a84118ecb834/manager/0.log" Jul 14 12:05:41.188692 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:41.188665 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" Jul 14 12:05:41.189067 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:41.188665 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j" event={"ID":"2af99f0f-534f-40f0-8ade-3f3cfe195634","Type":"ContainerDied","Data":"852834b8b62599316f2969f9ca0da63db7619bb65eeffe5cfbca7ed1679ec5c0"} Jul 14 12:05:41.189067 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:41.188811 2571 scope.go:117] "RemoveContainer" containerID="ce4d0df78954f1d01234288d137928ce4d5f7d6f197df2e9cc26a70c1eb730b1" Jul 14 12:05:41.220594 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:41.220571 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555b6fb65-qrcdx_24603c08-c875-4778-a6ee-bfbabc5d8cb0/copy-manifests/0.log" Jul 14 12:05:41.227667 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:41.227645 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j"] Jul 14 12:05:41.228287 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:41.228262 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555b6fb65-qrcdx_24603c08-c875-4778-a6ee-bfbabc5d8cb0/manager/0.log" Jul 14 12:05:41.232171 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:41.232151 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-0508a3-openshift-default-7d4998558c-9ww8j"] Jul 14 12:05:41.722744 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:41.722714 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-68f88d65bd-gkcsv_d6c90d15-7994-417e-8553-4631640569ea/maas-api/0.log" Jul 14 12:05:42.403567 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:42.403535 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2af99f0f-534f-40f0-8ade-3f3cfe195634" path="/var/lib/kubelet/pods/2af99f0f-534f-40f0-8ade-3f3cfe195634/volumes" Jul 14 12:05:44.399018 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:44.398982 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9_ef5afc8c-1590-43b6-b740-5768dd559481/util/0.log" Jul 14 12:05:44.407365 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:44.407345 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9_ef5afc8c-1590-43b6-b740-5768dd559481/pull/0.log" Jul 14 12:05:44.414508 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:44.414492 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9_ef5afc8c-1590-43b6-b740-5768dd559481/extract/0.log" Jul 14 12:05:44.528932 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:44.528903 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j_cc697cb2-85ad-4af1-b87f-7f30203b1bd2/util/0.log" Jul 14 12:05:44.538292 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:44.538271 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j_cc697cb2-85ad-4af1-b87f-7f30203b1bd2/pull/0.log" Jul 14 12:05:44.549864 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:44.549837 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j_cc697cb2-85ad-4af1-b87f-7f30203b1bd2/extract/0.log" Jul 14 12:05:44.662868 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:44.662798 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp_fbe24769-cda3-4f3a-a912-b13763402dd9/util/0.log" Jul 14 12:05:44.668904 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:44.668883 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp_fbe24769-cda3-4f3a-a912-b13763402dd9/pull/0.log" Jul 14 12:05:44.674574 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:44.674528 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp_fbe24769-cda3-4f3a-a912-b13763402dd9/extract/0.log" Jul 14 12:05:44.789258 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:44.789239 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j_5685eb77-c028-43e8-8532-20e60bf76588/util/0.log" Jul 14 12:05:44.795560 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:44.795544 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j_5685eb77-c028-43e8-8532-20e60bf76588/pull/0.log" Jul 14 12:05:44.801460 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:44.801443 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j_5685eb77-c028-43e8-8532-20e60bf76588/extract/0.log" Jul 14 12:05:46.175215 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:46.175187 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4lhp8r_e461ac53-05ac-4276-82fc-30f563afd714/istio-proxy/0.log" Jul 14 12:05:46.327456 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:46.327426 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qhzl2_acda5f77-8b32-4605-b298-ea11ae9e3f64/discovery/0.log" Jul 14 12:05:46.695061 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:46.695029 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-llkzr_6a4abd65-ce9d-4eec-a797-f4762b7538aa/istio-proxy/1.log" Jul 14 12:05:47.513528 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:47.513493 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-j4tj7_c3acafd6-f847-428b-a64f-dd4512bab0c1/main/0.log" Jul 14 12:05:47.622620 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:47.622589 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-jzclf_99d7dc85-274d-48da-812b-e523fffffc00/main/0.log" Jul 14 12:05:47.735010 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:47.734977 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbwwcmc_841a5533-4023-4b6e-aeec-e8c1aaa74120/main/0.log" Jul 14 12:05:51.906114 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:51.906075 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:05:51.982126 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:51.982077 2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" limit=1024 Jul 14 12:05:54.908001 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:54.907961 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-882pd_0fc0a645-d7c2-459a-8ecb-b0bae335b9ca/global-pull-secret-syncer/0.log" Jul 14 12:05:55.010369 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:55.010343 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-62zzs_3a64b363-5cc2-41ff-8a37-3df9b572ea88/konnectivity-agent/0.log" Jul 14 12:05:55.114021 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:55.113996 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-87.ec2.internal_90fe358c1daa74a54db164d6ac35b548/haproxy/0.log" Jul 14 12:05:59.458649 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:59.458609 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9_ef5afc8c-1590-43b6-b740-5768dd559481/extract/0.log" Jul 14 12:05:59.484867 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:59.484842 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9_ef5afc8c-1590-43b6-b740-5768dd559481/util/0.log" Jul 14 12:05:59.511639 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:59.511613 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595gwp9_ef5afc8c-1590-43b6-b740-5768dd559481/pull/0.log" Jul 14 12:05:59.542602 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:59.542584 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j_cc697cb2-85ad-4af1-b87f-7f30203b1bd2/extract/0.log" Jul 14 12:05:59.569544 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:59.569525 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j_cc697cb2-85ad-4af1-b87f-7f30203b1bd2/util/0.log" Jul 14 12:05:59.601270 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:59.601251 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0l2m2j_cc697cb2-85ad-4af1-b87f-7f30203b1bd2/pull/0.log" Jul 14 12:05:59.633422 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:59.633387 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp_fbe24769-cda3-4f3a-a912-b13763402dd9/extract/0.log" Jul 14 12:05:59.658439 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:59.658423 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp_fbe24769-cda3-4f3a-a912-b13763402dd9/util/0.log" Jul 14 12:05:59.683888 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:59.683866 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736dvgp_fbe24769-cda3-4f3a-a912-b13763402dd9/pull/0.log" Jul 14 12:05:59.714045 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:59.713977 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j_5685eb77-c028-43e8-8532-20e60bf76588/extract/0.log" Jul 14 12:05:59.737940 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:59.737923 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j_5685eb77-c028-43e8-8532-20e60bf76588/util/0.log" Jul 14 12:05:59.766694 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:05:59.766676 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1jtm2j_5685eb77-c028-43e8-8532-20e60bf76588/pull/0.log" Jul 14 12:06:02.345691 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:02.345668 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4d8fb55f-acfd-4179-a638-bf87cfb05545/alertmanager/0.log" Jul 14 12:06:02.378989 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:02.378968 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4d8fb55f-acfd-4179-a638-bf87cfb05545/config-reloader/0.log" Jul 14 12:06:02.406692 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:02.406670 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4d8fb55f-acfd-4179-a638-bf87cfb05545/kube-rbac-proxy-web/0.log" Jul 14 12:06:02.432918 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:02.432897 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4d8fb55f-acfd-4179-a638-bf87cfb05545/kube-rbac-proxy/0.log" Jul 14 12:06:02.456464 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:02.456442 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4d8fb55f-acfd-4179-a638-bf87cfb05545/kube-rbac-proxy-metric/0.log" Jul 14 12:06:02.479308 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:02.479280 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4d8fb55f-acfd-4179-a638-bf87cfb05545/prom-label-proxy/0.log" Jul 14 12:06:02.504399 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:02.504382 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_4d8fb55f-acfd-4179-a638-bf87cfb05545/init-config-reloader/0.log" Jul 14 12:06:02.624035 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:02.623985 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-9g65d_4f499a28-6bfb-4628-9b11-3553c93a1a7b/kube-state-metrics/0.log" Jul 14 12:06:02.648379 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:02.648360 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-9g65d_4f499a28-6bfb-4628-9b11-3553c93a1a7b/kube-rbac-proxy-main/0.log" Jul 14 12:06:02.671914 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:02.671895 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-9g65d_4f499a28-6bfb-4628-9b11-3553c93a1a7b/kube-rbac-proxy-self/0.log" Jul 14 12:06:02.702172 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:02.702152 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6f589fd54-c6cfb_1edfd667-50c9-4c39-9924-6770649f9c4a/metrics-server/0.log" Jul 14 12:06:02.849642 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:02.849611 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n48lw_d6f473b7-1178-4f03-b497-7f507c0d54ff/node-exporter/0.log" Jul 14 12:06:02.879668 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:02.879572 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n48lw_d6f473b7-1178-4f03-b497-7f507c0d54ff/kube-rbac-proxy/0.log" Jul 14 12:06:02.903359 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:02.903342 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n48lw_d6f473b7-1178-4f03-b497-7f507c0d54ff/init-textfile/0.log" Jul 14 12:06:03.313448 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:03.313408 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-f55mq_7221cd75-ecfe-4220-9716-a1ada0b79d76/prometheus-operator/0.log" Jul 14 12:06:03.335383 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:03.335360 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-f55mq_7221cd75-ecfe-4220-9716-a1ada0b79d76/kube-rbac-proxy/0.log" Jul 14 12:06:03.393590 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:03.393537 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85cb9b85bd-f7g4s_3475c11c-6707-4978-9b57-538d487b9806/telemeter-client/0.log" Jul 14 12:06:03.424270 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:03.424249 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85cb9b85bd-f7g4s_3475c11c-6707-4978-9b57-538d487b9806/reload/0.log" Jul 14 12:06:03.450084 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:03.450044 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85cb9b85bd-f7g4s_3475c11c-6707-4978-9b57-538d487b9806/kube-rbac-proxy/0.log" Jul 14 12:06:04.795119 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:04.795073 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-g6pxd_266bd207-c148-4d87-8eb7-baa4f02aa433/networking-console-plugin/0.log" Jul 14 12:06:05.029169 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.029053 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg"] Jul 14 12:06:05.029576 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.029544 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2af99f0f-534f-40f0-8ade-3f3cfe195634" containerName="istio-proxy" Jul 14 12:06:05.029576 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.029565 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2af99f0f-534f-40f0-8ade-3f3cfe195634" containerName="istio-proxy" Jul 14 12:06:05.029761 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.029584 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5216258f-a445-4253-81bd-10154d5ecf28" containerName="istio-proxy" Jul 14 12:06:05.029761 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.029593 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5216258f-a445-4253-81bd-10154d5ecf28" containerName="istio-proxy" Jul 14 12:06:05.029761 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.029613 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5216258f-a445-4253-81bd-10154d5ecf28" containerName="istio-proxy" Jul 14 12:06:05.029761 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.029619 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="5216258f-a445-4253-81bd-10154d5ecf28" containerName="istio-proxy" Jul 14 12:06:05.029761 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.029708 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2af99f0f-534f-40f0-8ade-3f3cfe195634" containerName="istio-proxy" Jul 14 12:06:05.029761 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.029719 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2af99f0f-534f-40f0-8ade-3f3cfe195634" containerName="istio-proxy" Jul 14 12:06:05.029761 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.029726 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="5216258f-a445-4253-81bd-10154d5ecf28" containerName="istio-proxy" Jul 14 12:06:05.032848 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.032831 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" Jul 14 12:06:05.035496 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.035478 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k5q25\"/\"kube-root-ca.crt\"" Jul 14 12:06:05.036688 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.036659 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-k5q25\"/\"default-dockercfg-m6pb5\"" Jul 14 12:06:05.036792 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.036695 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k5q25\"/\"openshift-service-ca.crt\"" Jul 14 12:06:05.043995 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.043973 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg"] Jul 14 12:06:05.133603 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.133575 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/27481cb2-62a1-45d5-96b6-38a0fd34ea7c-proc\") pod \"perf-node-gather-daemonset-d6hgg\" (UID: \"27481cb2-62a1-45d5-96b6-38a0fd34ea7c\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" Jul 14 12:06:05.133725 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.133608 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95vm9\" (UniqueName: \"kubernetes.io/projected/27481cb2-62a1-45d5-96b6-38a0fd34ea7c-kube-api-access-95vm9\") pod \"perf-node-gather-daemonset-d6hgg\" (UID: \"27481cb2-62a1-45d5-96b6-38a0fd34ea7c\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" Jul 14 12:06:05.133725 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.133673 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/27481cb2-62a1-45d5-96b6-38a0fd34ea7c-lib-modules\") pod \"perf-node-gather-daemonset-d6hgg\" (UID: \"27481cb2-62a1-45d5-96b6-38a0fd34ea7c\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" Jul 14 12:06:05.133725 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.133689 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/27481cb2-62a1-45d5-96b6-38a0fd34ea7c-podres\") pod \"perf-node-gather-daemonset-d6hgg\" (UID: \"27481cb2-62a1-45d5-96b6-38a0fd34ea7c\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" Jul 14 12:06:05.133849 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.133787 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27481cb2-62a1-45d5-96b6-38a0fd34ea7c-sys\") pod \"perf-node-gather-daemonset-d6hgg\" (UID: \"27481cb2-62a1-45d5-96b6-38a0fd34ea7c\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" Jul 14 12:06:05.234506 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.234482 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27481cb2-62a1-45d5-96b6-38a0fd34ea7c-sys\") pod \"perf-node-gather-daemonset-d6hgg\" (UID: \"27481cb2-62a1-45d5-96b6-38a0fd34ea7c\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" Jul 14 12:06:05.234617 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.234532 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/27481cb2-62a1-45d5-96b6-38a0fd34ea7c-proc\") pod \"perf-node-gather-daemonset-d6hgg\" (UID: \"27481cb2-62a1-45d5-96b6-38a0fd34ea7c\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" Jul 14 12:06:05.234617 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.234549 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95vm9\" (UniqueName: \"kubernetes.io/projected/27481cb2-62a1-45d5-96b6-38a0fd34ea7c-kube-api-access-95vm9\") pod \"perf-node-gather-daemonset-d6hgg\" (UID: \"27481cb2-62a1-45d5-96b6-38a0fd34ea7c\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" Jul 14 12:06:05.234617 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.234595 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27481cb2-62a1-45d5-96b6-38a0fd34ea7c-sys\") pod \"perf-node-gather-daemonset-d6hgg\" (UID: \"27481cb2-62a1-45d5-96b6-38a0fd34ea7c\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" Jul 14 12:06:05.234617 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.234608 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/27481cb2-62a1-45d5-96b6-38a0fd34ea7c-proc\") pod \"perf-node-gather-daemonset-d6hgg\" (UID: \"27481cb2-62a1-45d5-96b6-38a0fd34ea7c\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" Jul 14 12:06:05.234815 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.234650 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/27481cb2-62a1-45d5-96b6-38a0fd34ea7c-lib-modules\") pod \"perf-node-gather-daemonset-d6hgg\" (UID: \"27481cb2-62a1-45d5-96b6-38a0fd34ea7c\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" Jul 14 12:06:05.234815 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.234676 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/27481cb2-62a1-45d5-96b6-38a0fd34ea7c-podres\") pod \"perf-node-gather-daemonset-d6hgg\" (UID: \"27481cb2-62a1-45d5-96b6-38a0fd34ea7c\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" Jul 14 12:06:05.234815 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.234800 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/27481cb2-62a1-45d5-96b6-38a0fd34ea7c-podres\") pod \"perf-node-gather-daemonset-d6hgg\" (UID: \"27481cb2-62a1-45d5-96b6-38a0fd34ea7c\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" Jul 14 12:06:05.234815 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.234802 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/27481cb2-62a1-45d5-96b6-38a0fd34ea7c-lib-modules\") pod \"perf-node-gather-daemonset-d6hgg\" (UID: \"27481cb2-62a1-45d5-96b6-38a0fd34ea7c\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" Jul 14 12:06:05.243104 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.243085 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95vm9\" (UniqueName: \"kubernetes.io/projected/27481cb2-62a1-45d5-96b6-38a0fd34ea7c-kube-api-access-95vm9\") pod \"perf-node-gather-daemonset-d6hgg\" (UID: \"27481cb2-62a1-45d5-96b6-38a0fd34ea7c\") " pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" Jul 14 12:06:05.343917 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.343896 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" Jul 14 12:06:05.477455 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.477427 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg"] Jul 14 12:06:05.478592 ip-10-0-133-87 kubenswrapper[2571]: W0714 12:06:05.478558 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod27481cb2_62a1_45d5_96b6_38a0fd34ea7c.slice/crio-ea211dfdb5b044eb66ecc6262f13888015f751bbd2692799a47e93b5e1525c07 WatchSource:0}: Error finding container ea211dfdb5b044eb66ecc6262f13888015f751bbd2692799a47e93b5e1525c07: Status 404 returned error can't find the container with id ea211dfdb5b044eb66ecc6262f13888015f751bbd2692799a47e93b5e1525c07 Jul 14 12:06:05.968448 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:05.968419 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-79dbc94c75-fvgsh_3e8ba38c-6de8-4040-8a62-15d7dda90c88/console/0.log" Jul 14 12:06:06.001815 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:06.001784 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-9h6fb_0fe38157-e3ad-4fdb-a833-8418983f208f/download-server/0.log" Jul 14 12:06:06.323142 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:06.323050 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" event={"ID":"27481cb2-62a1-45d5-96b6-38a0fd34ea7c","Type":"ContainerStarted","Data":"674d8ad7a74526fe0379e00714d128a48a50e4701b199679439e3731f84a30a5"} Jul 14 12:06:06.323142 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:06.323101 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" event={"ID":"27481cb2-62a1-45d5-96b6-38a0fd34ea7c","Type":"ContainerStarted","Data":"ea211dfdb5b044eb66ecc6262f13888015f751bbd2692799a47e93b5e1525c07"} Jul 14 12:06:06.323352 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:06.323197 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" Jul 14 12:06:06.340032 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:06.339988 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" podStartSLOduration=1.33997469 podStartE2EDuration="1.33997469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 12:06:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 12:06:06.339667007 +0000 UTC m=+2860.500743475" watchObservedRunningTime="2026-07-14 12:06:06.33997469 +0000 UTC m=+2860.501051185" Jul 14 12:06:07.651953 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:07.651927 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9lqh6_6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5/dns/0.log" Jul 14 12:06:07.675518 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:07.675500 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9lqh6_6b1b3389-fa4a-4d72-b7f6-ffc78326fbc5/kube-rbac-proxy/0.log" Jul 14 12:06:07.856448 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:07.856423 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ttsdl_fa67ad6b-3e7d-4fb4-9f0e-c186cfa8912a/dns-node-resolver/0.log" Jul 14 12:06:08.481557 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:08.481523 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zmxv2_b27010c7-44aa-40f5-8aa3-ae1c20e6c7f4/node-ca/0.log" Jul 14 12:06:09.462290 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:09.462259 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4lhp8r_e461ac53-05ac-4276-82fc-30f563afd714/istio-proxy/0.log" Jul 14 12:06:09.605190 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:09.605155 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qhzl2_acda5f77-8b32-4605-b298-ea11ae9e3f64/discovery/0.log" Jul 14 12:06:09.739607 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:09.739527 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-llkzr_6a4abd65-ce9d-4eec-a797-f4762b7538aa/istio-proxy/0.log" Jul 14 12:06:09.767898 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:09.767873 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-llkzr_6a4abd65-ce9d-4eec-a797-f4762b7538aa/istio-proxy/1.log" Jul 14 12:06:10.736966 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:10.736938 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z9xv5_e142c698-e9cd-4a19-943f-74132dd73b2f/serve-healthcheck-canary/0.log" Jul 14 12:06:11.366241 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:11.366216 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rc2bq_e85830b4-be15-4f4c-ab66-62ecc464628f/kube-rbac-proxy/0.log" Jul 14 12:06:11.389347 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:11.389319 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rc2bq_e85830b4-be15-4f4c-ab66-62ecc464628f/exporter/0.log" Jul 14 12:06:11.413593 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:11.413572 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rc2bq_e85830b4-be15-4f4c-ab66-62ecc464628f/extractor/0.log" Jul 14 12:06:12.338827 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:12.338799 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-k5q25/perf-node-gather-daemonset-d6hgg" Jul 14 12:06:14.211842 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:14.211818 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5f86cb7fff-gcb99_29adf428-76cd-41cd-9ee6-a84118ecb834/manager/0.log" Jul 14 12:06:14.446707 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:14.446687 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555b6fb65-qrcdx_24603c08-c875-4778-a6ee-bfbabc5d8cb0/manager/0.log" Jul 14 12:06:14.471129 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:14.471108 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555b6fb65-qrcdx_24603c08-c875-4778-a6ee-bfbabc5d8cb0/copy-manifests/0.log" Jul 14 12:06:15.218518 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:15.218492 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4lhp8r_e461ac53-05ac-4276-82fc-30f563afd714/istio-proxy/0.log" Jul 14 12:06:15.321694 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:15.321642 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qhzl2_acda5f77-8b32-4605-b298-ea11ae9e3f64/discovery/0.log" Jul 14 12:06:15.369411 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:15.369368 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-llkzr_6a4abd65-ce9d-4eec-a797-f4762b7538aa/istio-proxy/0.log" Jul 14 12:06:15.398661 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:15.398616 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-llkzr_6a4abd65-ce9d-4eec-a797-f4762b7538aa/istio-proxy/1.log" Jul 14 12:06:15.903323 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:15.903299 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-985446bdb-t76xq_1f8daf19-7942-4073-8134-50e76918a5ca/manager/0.log" Jul 14 12:06:16.729336 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:16.729300 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4lhp8r_e461ac53-05ac-4276-82fc-30f563afd714/istio-proxy/0.log" Jul 14 12:06:16.839130 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:16.839098 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qhzl2_acda5f77-8b32-4605-b298-ea11ae9e3f64/discovery/0.log" Jul 14 12:06:16.896453 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:16.896386 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-llkzr_6a4abd65-ce9d-4eec-a797-f4762b7538aa/istio-proxy/0.log" Jul 14 12:06:16.925737 ip-10-0-133-87 kubenswrapper[2571]: I0714 12:06:16.925713 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-llkzr_6a4abd65-ce9d-4eec-a797-f4762b7538aa/istio-proxy/1.log"