Jul 29 14:35:21.940439 ip-10-0-132-42 systemd[1]: Starting Kubernetes Kubelet... Jul 29 14:35:22.361433 ip-10-0-132-42 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 14:35:22.361433 ip-10-0-132-42 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 29 14:35:22.361433 ip-10-0-132-42 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 14:35:22.361433 ip-10-0-132-42 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 29 14:35:22.361910 ip-10-0-132-42 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 14:35:22.361910 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.361532 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 29 14:35:22.366616 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366601 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 14:35:22.366616 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366615 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 14:35:22.366686 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366621 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 14:35:22.366686 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366624 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 14:35:22.366686 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366627 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 14:35:22.366686 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366631 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 14:35:22.366686 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366634 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 14:35:22.366686 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366636 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 14:35:22.366686 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366639 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 14:35:22.366686 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366641 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 14:35:22.366686 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366644 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 14:35:22.366686 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366646 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 14:35:22.366686 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366649 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 14:35:22.366686 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366651 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 14:35:22.366686 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366653 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 14:35:22.366686 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366656 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 14:35:22.366686 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366658 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 14:35:22.366686 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366661 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 14:35:22.366686 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366663 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 14:35:22.366686 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366666 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 14:35:22.366686 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366668 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 14:35:22.366686 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366671 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 14:35:22.367143 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366673 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 14:35:22.367143 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366676 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 14:35:22.367143 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366679 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 14:35:22.367143 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366681 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 14:35:22.367143 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366683 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 14:35:22.367143 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366686 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 14:35:22.367143 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366688 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 14:35:22.367143 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366691 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 14:35:22.367143 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366694 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 14:35:22.367143 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366697 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 14:35:22.367143 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366700 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 29 14:35:22.367143 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366702 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 14:35:22.367143 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366705 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 14:35:22.367143 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366707 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 14:35:22.367143 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366710 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 14:35:22.367143 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366712 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 14:35:22.367143 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366715 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 14:35:22.367143 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366717 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 14:35:22.367143 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366720 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 14:35:22.367143 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366722 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 14:35:22.367647 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366724 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 14:35:22.367647 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366727 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 14:35:22.367647 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366729 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 14:35:22.367647 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366732 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 14:35:22.367647 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366734 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 14:35:22.367647 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366736 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 14:35:22.367647 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366739 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 14:35:22.367647 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366741 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 14:35:22.367647 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366744 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 14:35:22.367647 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366746 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 14:35:22.367647 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366748 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 14:35:22.367647 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366751 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 14:35:22.367647 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366754 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 14:35:22.367647 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366756 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 14:35:22.367647 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366759 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 14:35:22.367647 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366762 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 14:35:22.367647 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366764 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 14:35:22.367647 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366767 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 14:35:22.367647 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366770 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 14:35:22.367647 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366772 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 14:35:22.368231 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366775 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 14:35:22.368231 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366777 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 14:35:22.368231 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366780 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 14:35:22.368231 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366783 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 14:35:22.368231 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366785 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 14:35:22.368231 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366787 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 14:35:22.368231 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366790 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 14:35:22.368231 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366794 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 14:35:22.368231 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366798 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 14:35:22.368231 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366801 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 14:35:22.368231 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366803 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 14:35:22.368231 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366806 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 14:35:22.368231 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366808 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 14:35:22.368231 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366810 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 14:35:22.368231 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366813 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 14:35:22.368231 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366816 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 14:35:22.368231 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366818 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 14:35:22.368231 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366821 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 14:35:22.368231 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366823 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 14:35:22.368960 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366826 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 14:35:22.368960 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366828 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 14:35:22.368960 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366831 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 14:35:22.368960 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366833 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 14:35:22.368960 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366836 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 14:35:22.368960 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366840 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 14:35:22.368960 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366842 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 14:35:22.368960 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366845 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 14:35:22.368960 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366849 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 14:35:22.368960 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366853 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 14:35:22.368960 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366856 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 14:35:22.368960 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366859 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 14:35:22.368960 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366861 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 14:35:22.368960 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366865 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 14:35:22.368960 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366868 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 14:35:22.368960 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366871 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 14:35:22.368960 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366874 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 14:35:22.368960 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.366877 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 14:35:22.368960 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367565 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 14:35:22.369439 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367573 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 14:35:22.369439 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367576 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 14:35:22.369439 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367579 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 14:35:22.369439 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367582 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 14:35:22.369439 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367585 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 14:35:22.369439 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367589 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 14:35:22.369439 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367592 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 14:35:22.369439 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367594 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 14:35:22.369439 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367597 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 14:35:22.369439 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367600 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 14:35:22.369439 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367603 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 14:35:22.369439 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367605 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 14:35:22.369439 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367608 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 14:35:22.369439 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367610 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 14:35:22.369439 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367613 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 14:35:22.369439 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367615 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 14:35:22.369439 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367618 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 14:35:22.369439 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367620 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 14:35:22.369439 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367623 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 14:35:22.369439 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367626 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 14:35:22.369929 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367628 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 14:35:22.369929 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367631 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 14:35:22.369929 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367633 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 14:35:22.369929 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367636 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 14:35:22.369929 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367638 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 14:35:22.369929 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367641 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 14:35:22.369929 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367643 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 14:35:22.369929 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367646 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 14:35:22.369929 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367649 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 14:35:22.369929 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367652 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 14:35:22.369929 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367654 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 14:35:22.369929 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367657 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 29 14:35:22.369929 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367659 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 14:35:22.369929 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367662 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 14:35:22.369929 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367665 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 14:35:22.369929 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367667 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 14:35:22.369929 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367669 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 14:35:22.369929 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367672 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 14:35:22.369929 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367675 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 14:35:22.369929 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367677 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 14:35:22.370422 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367680 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 14:35:22.370422 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367682 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 14:35:22.370422 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367685 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 14:35:22.370422 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367688 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 14:35:22.370422 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367691 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 14:35:22.370422 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367693 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 14:35:22.370422 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367695 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 14:35:22.370422 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367697 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 14:35:22.370422 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367700 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 14:35:22.370422 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367702 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 14:35:22.370422 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367705 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 14:35:22.370422 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367707 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 14:35:22.370422 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367711 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 14:35:22.370422 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367715 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 14:35:22.370422 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367718 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 14:35:22.370422 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367720 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 14:35:22.370422 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367723 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 14:35:22.370422 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367726 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 14:35:22.370422 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367728 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 14:35:22.370873 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367731 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 14:35:22.370873 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367733 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 14:35:22.370873 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367736 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 14:35:22.370873 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367738 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 14:35:22.370873 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367740 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 14:35:22.370873 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367743 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 14:35:22.370873 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367746 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 14:35:22.370873 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367748 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 14:35:22.370873 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367751 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 14:35:22.370873 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367753 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 14:35:22.370873 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367755 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 14:35:22.370873 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367759 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 14:35:22.370873 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367762 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 14:35:22.370873 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367765 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 14:35:22.370873 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367769 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 14:35:22.370873 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367773 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 14:35:22.370873 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367776 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 14:35:22.370873 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367778 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 14:35:22.370873 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367781 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 14:35:22.370873 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367783 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 14:35:22.371374 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367786 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 14:35:22.371374 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367788 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 14:35:22.371374 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367790 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 14:35:22.371374 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367794 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 14:35:22.371374 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367796 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 14:35:22.371374 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367799 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 14:35:22.371374 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367801 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 14:35:22.371374 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367804 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 14:35:22.371374 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367806 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 14:35:22.371374 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367808 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 14:35:22.371374 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367811 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 14:35:22.371374 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367814 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 14:35:22.371374 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367817 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 14:35:22.371374 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367819 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 14:35:22.371374 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367821 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 14:35:22.371374 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367825 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 14:35:22.371374 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367828 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 14:35:22.371374 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367831 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 14:35:22.371374 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.367833 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 14:35:22.371824 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370452 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 14:35:22.371824 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370463 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 14:35:22.371824 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370466 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 14:35:22.371824 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370469 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 14:35:22.371824 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370472 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 14:35:22.371824 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370475 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 14:35:22.371824 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370478 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 14:35:22.371824 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370481 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 14:35:22.371824 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370484 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 14:35:22.371824 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370487 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 14:35:22.371824 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370491 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 14:35:22.371824 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370494 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 29 14:35:22.371824 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370497 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 14:35:22.371824 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370501 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 14:35:22.371824 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370503 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 14:35:22.371824 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370506 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 14:35:22.371824 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370509 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 14:35:22.371824 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370511 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 14:35:22.371824 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370514 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 14:35:22.372319 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370518 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 14:35:22.372319 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370520 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 14:35:22.372319 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370523 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 14:35:22.372319 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370525 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 14:35:22.372319 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370528 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 14:35:22.372319 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370530 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 14:35:22.372319 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370533 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 14:35:22.372319 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370535 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 14:35:22.372319 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370539 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 14:35:22.372319 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370543 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 14:35:22.372319 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370547 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 14:35:22.372319 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370549 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 14:35:22.372319 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370557 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 14:35:22.372319 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370561 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 14:35:22.372319 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370564 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 14:35:22.372319 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370567 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 14:35:22.372319 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370569 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 14:35:22.372319 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370572 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 14:35:22.372319 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370575 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 14:35:22.372764 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370579 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 14:35:22.372764 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370583 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 14:35:22.372764 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370586 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 14:35:22.372764 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370589 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 14:35:22.372764 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370592 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 14:35:22.372764 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370594 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 14:35:22.372764 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370597 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 14:35:22.372764 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370600 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 14:35:22.372764 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370602 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 14:35:22.372764 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370604 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 14:35:22.372764 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370607 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 14:35:22.372764 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370609 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 14:35:22.372764 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370619 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 14:35:22.372764 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370621 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 14:35:22.372764 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370624 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 14:35:22.372764 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370626 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 14:35:22.372764 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370629 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 14:35:22.372764 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370631 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 14:35:22.372764 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370634 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 14:35:22.372764 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370636 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 14:35:22.373281 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370639 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 14:35:22.373281 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370641 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 14:35:22.373281 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370643 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 14:35:22.373281 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370646 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 14:35:22.373281 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370648 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 14:35:22.373281 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370650 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 14:35:22.373281 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370653 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 14:35:22.373281 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370656 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 14:35:22.373281 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370658 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 14:35:22.373281 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370661 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 14:35:22.373281 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370663 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 14:35:22.373281 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370666 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 14:35:22.373281 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370668 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 14:35:22.373281 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370671 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 14:35:22.373281 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370673 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 14:35:22.373281 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370676 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 14:35:22.373281 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370678 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 14:35:22.373281 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370681 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 14:35:22.373281 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370683 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 14:35:22.373281 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370685 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 14:35:22.373786 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370688 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 14:35:22.373786 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370690 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 14:35:22.373786 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370693 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 14:35:22.373786 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370695 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 14:35:22.373786 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370698 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 14:35:22.373786 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370705 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 14:35:22.373786 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370708 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 14:35:22.373786 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370710 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 14:35:22.373786 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370713 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 14:35:22.373786 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370715 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 14:35:22.373786 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370718 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 14:35:22.373786 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370720 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 14:35:22.373786 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370725 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 14:35:22.373786 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370727 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 14:35:22.373786 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370730 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 14:35:22.373786 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370732 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 14:35:22.373786 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370735 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 14:35:22.373786 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370737 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 14:35:22.373786 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370740 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 14:35:22.373786 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370743 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.370745 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370809 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370815 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370827 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370831 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370835 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370838 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370842 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370847 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370850 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370852 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370856 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370859 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370862 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370864 2580 flags.go:64] FLAG: --cgroup-root="" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370867 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370870 2580 flags.go:64] FLAG: --client-ca-file="" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370873 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370881 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370904 2580 flags.go:64] FLAG: --cluster-domain="" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370907 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370911 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370914 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 29 14:35:22.374261 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370917 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370921 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370924 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370927 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370930 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370933 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370936 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370939 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370942 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370945 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370948 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370951 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370954 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370956 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370959 2580 flags.go:64] FLAG: --enable-server="true" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370962 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370969 2580 flags.go:64] FLAG: --event-burst="100" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370972 2580 flags.go:64] FLAG: --event-qps="50" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370975 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370978 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370980 2580 flags.go:64] FLAG: --eviction-hard="" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370988 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370991 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370994 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.370997 2580 flags.go:64] FLAG: --eviction-soft="" Jul 29 14:35:22.374843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371000 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371002 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371005 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371013 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371016 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371019 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371023 2580 flags.go:64] FLAG: --feature-gates="" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371026 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371029 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371032 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371035 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371038 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371040 2580 flags.go:64] FLAG: --help="false" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371043 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-132-42.ec2.internal" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371049 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371052 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371055 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371058 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371061 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371064 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371067 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371070 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371074 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 29 14:35:22.375457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371077 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371080 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371083 2580 flags.go:64] FLAG: --kube-reserved="" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371086 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371088 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371091 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371094 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371097 2580 flags.go:64] FLAG: --lock-file="" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371099 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371102 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371105 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371110 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371113 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371121 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371124 2580 flags.go:64] FLAG: --logging-format="text" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371127 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371130 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371132 2580 flags.go:64] FLAG: --manifest-url="" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371135 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371139 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371142 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371146 2580 flags.go:64] FLAG: --max-pods="110" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371149 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371153 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371156 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 29 14:35:22.376027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371159 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371162 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371165 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371167 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371173 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371176 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371180 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371200 2580 flags.go:64] FLAG: --pod-cidr="" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371203 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371209 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371212 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371215 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371217 2580 flags.go:64] FLAG: --port="10250" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371220 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371223 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0bc6f71141634b7fd" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371226 2580 flags.go:64] FLAG: --qos-reserved="" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371229 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371232 2580 flags.go:64] FLAG: --register-node="true" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371235 2580 flags.go:64] FLAG: --register-with-taints="" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371238 2580 flags.go:64] FLAG: --registry-burst="10" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371241 2580 flags.go:64] FLAG: --registry-qps="5" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371250 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371253 2580 flags.go:64] FLAG: --reserved-memory="" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371257 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371260 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 29 14:35:22.376650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371263 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371265 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371268 2580 flags.go:64] FLAG: --runonce="false" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371271 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371274 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371276 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371280 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371283 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371286 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371289 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371292 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371295 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371298 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371301 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371304 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371307 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371310 2580 flags.go:64] FLAG: --system-cgroups="" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371313 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371318 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371321 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371324 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371330 2580 flags.go:64] FLAG: --tls-min-version="" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371333 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371335 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371338 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 29 14:35:22.377411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371341 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 29 14:35:22.378456 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371344 2580 flags.go:64] FLAG: --v="2" Jul 29 14:35:22.378456 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371347 2580 flags.go:64] FLAG: --version="false" Jul 29 14:35:22.378456 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371351 2580 flags.go:64] FLAG: --vmodule="" Jul 29 14:35:22.378456 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371361 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 29 14:35:22.378456 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.371364 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 29 14:35:22.378456 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371478 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 14:35:22.378456 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371483 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 14:35:22.378456 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371487 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 14:35:22.378456 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371490 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 14:35:22.378456 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371493 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 14:35:22.378456 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371496 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 14:35:22.378456 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371499 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 14:35:22.378456 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371504 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 14:35:22.378456 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371507 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 14:35:22.378456 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371509 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 14:35:22.378456 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371512 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 14:35:22.378456 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371515 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 14:35:22.378456 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371517 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 14:35:22.378456 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371520 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 14:35:22.378456 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371524 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 14:35:22.378456 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371526 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 14:35:22.379351 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371529 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 14:35:22.379351 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371531 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 14:35:22.379351 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371534 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 14:35:22.379351 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371536 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 14:35:22.379351 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371539 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 14:35:22.379351 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371541 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 14:35:22.379351 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371544 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 14:35:22.379351 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371546 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 14:35:22.379351 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371549 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 14:35:22.379351 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371551 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 14:35:22.379351 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371553 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 14:35:22.379351 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371556 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 14:35:22.379351 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371558 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 14:35:22.379351 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371561 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 14:35:22.379351 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371563 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 14:35:22.379351 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371571 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 14:35:22.379351 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371574 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 14:35:22.379351 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371576 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 14:35:22.379351 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371579 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 14:35:22.379351 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371581 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 14:35:22.380172 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371584 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 14:35:22.380172 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371586 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 14:35:22.380172 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371589 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 14:35:22.380172 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371592 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 14:35:22.380172 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371595 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 14:35:22.380172 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371597 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 14:35:22.380172 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371600 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 14:35:22.380172 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371603 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 14:35:22.380172 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371607 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 29 14:35:22.380172 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371610 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 14:35:22.380172 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371614 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 14:35:22.380172 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371617 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 14:35:22.380172 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371619 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 14:35:22.380172 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371622 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 14:35:22.380172 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371624 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 14:35:22.380172 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371627 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 14:35:22.380172 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371630 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 14:35:22.380172 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371632 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 14:35:22.380172 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371635 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 14:35:22.380924 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371637 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 14:35:22.380924 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371640 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 14:35:22.380924 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371642 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 14:35:22.380924 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371645 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 14:35:22.380924 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371647 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 14:35:22.380924 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371650 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 14:35:22.380924 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371652 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 14:35:22.380924 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371655 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 14:35:22.380924 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371658 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 14:35:22.380924 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371669 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 14:35:22.380924 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371672 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 14:35:22.380924 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371674 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 14:35:22.380924 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371677 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 14:35:22.380924 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371679 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 14:35:22.380924 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371682 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 14:35:22.380924 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371684 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 14:35:22.380924 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371688 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 14:35:22.380924 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371692 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 14:35:22.380924 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371695 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 14:35:22.381602 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371698 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 14:35:22.381602 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371701 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 14:35:22.381602 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371704 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 14:35:22.381602 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371707 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 14:35:22.381602 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371711 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 14:35:22.381602 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371721 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 14:35:22.381602 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371724 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 14:35:22.381602 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371727 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 14:35:22.381602 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371730 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 14:35:22.381602 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371733 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 14:35:22.381602 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371736 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 14:35:22.381602 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371739 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 14:35:22.381602 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371742 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 14:35:22.381602 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371745 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 14:35:22.381602 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371747 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 14:35:22.381602 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371750 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 14:35:22.381602 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371752 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 14:35:22.381602 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371755 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 14:35:22.381602 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371757 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 14:35:22.382405 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371760 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 14:35:22.382405 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371762 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 14:35:22.382405 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371764 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 14:35:22.382405 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371767 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 14:35:22.382405 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371775 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 14:35:22.382405 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.371777 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 14:35:22.382405 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.372643 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 14:35:22.382405 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.379282 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 29 14:35:22.382405 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.379300 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 29 14:35:22.382405 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.379318 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 29 14:35:22.382405 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.379331 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 29 14:35:22.382405 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379417 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 14:35:22.382405 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379425 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 14:35:22.382405 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379430 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 14:35:22.382405 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379435 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 14:35:22.382883 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379439 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 14:35:22.382883 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379443 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 14:35:22.382883 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379448 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 14:35:22.382883 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379453 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 14:35:22.382883 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379457 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 14:35:22.382883 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379462 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 14:35:22.382883 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379466 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 14:35:22.382883 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379471 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 14:35:22.382883 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379475 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 14:35:22.382883 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379481 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 14:35:22.382883 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379485 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 14:35:22.382883 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379490 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 14:35:22.382883 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379494 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 14:35:22.382883 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379499 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 14:35:22.382883 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379504 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 14:35:22.382883 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379508 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 14:35:22.382883 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379513 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 14:35:22.382883 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379517 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 14:35:22.382883 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379521 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 14:35:22.382883 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379525 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 14:35:22.382883 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379529 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 14:35:22.383525 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379536 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 14:35:22.383525 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379542 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 14:35:22.383525 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379547 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 14:35:22.383525 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379552 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 14:35:22.383525 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379558 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 14:35:22.383525 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379563 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 14:35:22.383525 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379567 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 14:35:22.383525 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379572 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 14:35:22.383525 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379576 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 14:35:22.383525 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379581 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 14:35:22.383525 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379586 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 14:35:22.383525 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379590 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 14:35:22.383525 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379594 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 14:35:22.383525 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379598 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 14:35:22.383525 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379603 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 14:35:22.383525 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379609 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 14:35:22.383525 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379614 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 14:35:22.383525 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379618 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 14:35:22.383525 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379622 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 14:35:22.384003 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379627 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 14:35:22.384003 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379631 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 14:35:22.384003 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379635 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 14:35:22.384003 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379639 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 14:35:22.384003 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379643 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 14:35:22.384003 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379647 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 14:35:22.384003 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379651 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 14:35:22.384003 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379656 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 14:35:22.384003 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379660 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 14:35:22.384003 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379664 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 14:35:22.384003 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379668 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 14:35:22.384003 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379673 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 14:35:22.384003 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379677 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 14:35:22.384003 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379681 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 14:35:22.384003 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379685 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 14:35:22.384003 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379689 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 14:35:22.384003 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379693 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 14:35:22.384003 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379698 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 14:35:22.384003 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379703 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 14:35:22.384003 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379708 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 29 14:35:22.384507 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379714 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 14:35:22.384507 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379719 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 14:35:22.384507 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379724 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 14:35:22.384507 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379729 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 14:35:22.384507 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379733 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 14:35:22.384507 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379737 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 14:35:22.384507 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379742 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 14:35:22.384507 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379746 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 14:35:22.384507 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379751 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 14:35:22.384507 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379755 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 14:35:22.384507 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379759 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 14:35:22.384507 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379763 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 14:35:22.384507 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379767 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 14:35:22.384507 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379771 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 14:35:22.384507 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379775 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 14:35:22.384507 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379779 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 14:35:22.384507 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379783 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 14:35:22.384507 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379787 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 14:35:22.384507 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379791 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 14:35:22.384998 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379796 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 14:35:22.384998 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379800 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 14:35:22.384998 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379804 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 14:35:22.384998 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379808 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 14:35:22.384998 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379812 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 14:35:22.384998 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379815 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 14:35:22.384998 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379819 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 14:35:22.384998 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379824 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 14:35:22.384998 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379828 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 14:35:22.384998 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379833 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 14:35:22.384998 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379837 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 14:35:22.384998 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379841 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 14:35:22.384998 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379846 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 14:35:22.384998 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379850 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 14:35:22.384998 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379855 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 14:35:22.384998 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.379859 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 14:35:22.385447 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.379866 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 14:35:22.385447 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380041 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 14:35:22.385447 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380049 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 14:35:22.385447 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380055 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 14:35:22.385447 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380060 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 14:35:22.385447 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380064 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 14:35:22.385447 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380068 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 14:35:22.385447 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380073 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 14:35:22.385447 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380077 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 14:35:22.385447 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380081 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 14:35:22.385447 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380085 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 14:35:22.385447 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380089 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 14:35:22.385447 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380093 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 14:35:22.385447 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380097 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 14:35:22.385447 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380101 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 14:35:22.385835 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380106 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 14:35:22.385835 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380110 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 14:35:22.385835 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380114 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 14:35:22.385835 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380118 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 14:35:22.385835 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380122 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 14:35:22.385835 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380126 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 14:35:22.385835 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380130 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 14:35:22.385835 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380134 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 14:35:22.385835 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380138 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 14:35:22.385835 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380142 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 14:35:22.385835 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380146 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 14:35:22.385835 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380151 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 14:35:22.385835 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380155 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 14:35:22.385835 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380160 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 14:35:22.385835 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380165 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 14:35:22.385835 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380169 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 14:35:22.385835 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380175 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 14:35:22.385835 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380180 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 14:35:22.385835 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380201 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 14:35:22.385835 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380205 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 14:35:22.386348 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380209 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 14:35:22.386348 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380212 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 14:35:22.386348 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380216 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 14:35:22.386348 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380219 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 14:35:22.386348 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380223 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 14:35:22.386348 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380227 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 29 14:35:22.386348 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380231 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 14:35:22.386348 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380235 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 14:35:22.386348 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380238 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 14:35:22.386348 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380242 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 14:35:22.386348 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380246 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 14:35:22.386348 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380250 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 14:35:22.386348 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380253 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 14:35:22.386348 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380257 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 14:35:22.386348 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380261 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 14:35:22.386348 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380265 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 14:35:22.386348 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380269 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 14:35:22.386348 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380273 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 14:35:22.386348 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380277 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 14:35:22.386348 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380281 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 14:35:22.386888 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380285 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 14:35:22.386888 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380289 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 14:35:22.386888 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380293 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 14:35:22.386888 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380297 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 14:35:22.386888 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380301 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 14:35:22.386888 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380305 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 14:35:22.386888 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380309 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 14:35:22.386888 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380318 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 14:35:22.386888 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380325 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 14:35:22.386888 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380329 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 14:35:22.386888 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380335 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 14:35:22.386888 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380339 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 14:35:22.386888 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380343 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 14:35:22.386888 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380347 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 14:35:22.386888 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380351 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 14:35:22.386888 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380355 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 14:35:22.386888 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380359 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 14:35:22.386888 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380363 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 14:35:22.386888 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380367 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 14:35:22.386888 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380371 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 14:35:22.387691 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380375 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 14:35:22.387691 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380380 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 14:35:22.387691 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380384 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 14:35:22.387691 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380388 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 14:35:22.387691 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380394 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 14:35:22.387691 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380400 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 14:35:22.387691 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380404 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 14:35:22.387691 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380409 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 14:35:22.387691 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380414 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 14:35:22.387691 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380418 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 14:35:22.387691 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380422 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 14:35:22.387691 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380427 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 14:35:22.387691 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380431 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 14:35:22.387691 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380435 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 14:35:22.387691 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380439 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 14:35:22.387691 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380443 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 14:35:22.387691 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380447 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 14:35:22.387691 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380451 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 14:35:22.387691 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380455 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 14:35:22.388366 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380460 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 14:35:22.388366 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380465 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 14:35:22.388366 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380469 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 14:35:22.388366 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380479 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 14:35:22.388366 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380483 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 14:35:22.388366 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:22.380487 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 14:35:22.388366 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.380495 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 14:35:22.388366 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.380649 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 29 14:35:22.388366 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.383218 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 29 14:35:22.388366 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.384628 2580 server.go:1019] "Starting client certificate rotation" Jul 29 14:35:22.388366 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.384754 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 29 14:35:22.388366 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.384780 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 29 14:35:22.388366 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.387422 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 29 14:35:22.389670 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.389653 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 29 14:35:22.402166 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.402149 2580 log.go:25] "Validated CRI v1 runtime API" Jul 29 14:35:22.404028 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.404005 2580 log.go:25] "Validated CRI v1 image API" Jul 29 14:35:22.406204 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.406177 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 29 14:35:22.409223 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.409198 2580 fs.go:135] Filesystem UUIDs: map[4b9341ee-b45d-47a6-aa8d-9a8fe7aa6591:/dev/nvme0n1p4 53ea3aca-bf5c-4353-816f-d8dba001859e:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 29 14:35:22.409276 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.409223 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 29 14:35:22.414406 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.414390 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 29 14:35:22.415786 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.415690 2580 manager.go:216] Machine: {Timestamp:2026-07-29 14:35:22.414250605 +0000 UTC m=+0.352936013 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3220290 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2aa745a42052a364ad716f904f15f7 SystemUUID:ec2aa745-a420-52a3-64ad-716f904f15f7 BootID:f5b3bdf5-6de0-49e3-9fad-1a9382e83c2a Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2e:7f:4b:07:15 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2e:7f:4b:07:15 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:ff:98:81:31:85 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 29 14:35:22.415786 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.415781 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 29 14:35:22.415894 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.415842 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 29 14:35:22.419279 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.419258 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 29 14:35:22.419397 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.419282 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-42.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 29 14:35:22.420852 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.420842 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 29 14:35:22.420882 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.420854 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 29 14:35:22.420882 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.420862 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 29 14:35:22.420882 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.420875 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 29 14:35:22.421611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.421601 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 29 14:35:22.423277 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.423266 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 29 14:35:22.423383 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.423374 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 29 14:35:22.423410 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.423404 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 29 14:35:22.423437 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.423415 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 29 14:35:22.423437 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.423432 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 29 14:35:22.423491 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.423441 2580 kubelet.go:398] "Adding apiserver pod source" Jul 29 14:35:22.423491 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.423449 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 29 14:35:22.423745 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.423724 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 29 14:35:22.423745 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.423747 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 29 14:35:22.425988 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.425971 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 29 14:35:22.427695 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.427682 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 29 14:35:22.427741 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.427711 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 29 14:35:22.429459 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.429448 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 29 14:35:22.429501 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.429467 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 29 14:35:22.429501 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.429478 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 29 14:35:22.429501 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.429485 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 29 14:35:22.429501 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.429493 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 29 14:35:22.429501 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.429498 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 29 14:35:22.429628 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.429506 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 29 14:35:22.429628 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.429511 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 29 14:35:22.429628 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.429517 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 29 14:35:22.429628 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.429522 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 29 14:35:22.429628 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.429536 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 29 14:35:22.429628 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.429541 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 29 14:35:22.430955 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.430943 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 29 14:35:22.430992 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.430955 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 29 14:35:22.434013 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.433998 2580 server.go:1268] "Started kubelet" Jul 29 14:35:22.434111 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.434084 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 29 14:35:22.434368 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.434098 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 29 14:35:22.434458 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.434397 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 29 14:35:22.434723 ip-10-0-132-42 systemd[1]: Started Kubernetes Kubelet. Jul 29 14:35:22.435031 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:22.434995 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 29 14:35:22.435031 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.435015 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-42.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 29 14:35:22.435031 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:22.434996 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-42.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 29 14:35:22.436302 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.436286 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 29 14:35:22.436624 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.436611 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 29 14:35:22.438229 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:22.438204 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 29 14:35:22.438512 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.438496 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sv76z" Jul 29 14:35:22.440854 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:22.439787 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-42.ec2.internal.18c6c961647e0634 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-42.ec2.internal,UID:ip-10-0-132-42.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-42.ec2.internal,},FirstTimestamp:2026-07-29 14:35:22.433971764 +0000 UTC m=+0.372657153,LastTimestamp:2026-07-29 14:35:22.433971764 +0000 UTC m=+0.372657153,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-42.ec2.internal,}" Jul 29 14:35:22.441667 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.441648 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 29 14:35:22.441765 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.441650 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 29 14:35:22.441765 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:22.441752 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-42.ec2.internal\" not found" Jul 29 14:35:22.442480 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.442450 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 29 14:35:22.442574 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.442482 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 29 14:35:22.442574 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.442570 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 29 14:35:22.443027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.443012 2580 factory.go:55] Registering systemd factory Jul 29 14:35:22.443107 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.443032 2580 factory.go:223] Registration of the systemd container factory successfully Jul 29 14:35:22.444044 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.444027 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 29 14:35:22.444044 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.444041 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 29 14:35:22.444441 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.444422 2580 factory.go:153] Registering CRI-O factory Jul 29 14:35:22.444524 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.444444 2580 factory.go:223] Registration of the crio container factory successfully Jul 29 14:35:22.444524 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.444492 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 29 14:35:22.444524 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.444514 2580 factory.go:103] Registering Raw factory Jul 29 14:35:22.444710 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.444529 2580 manager.go:1196] Started watching for new ooms in manager Jul 29 14:35:22.445006 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.444988 2580 manager.go:349] Starting recovery of all containers Jul 29 14:35:22.445568 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.445542 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sv76z" Jul 29 14:35:22.447472 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:22.447444 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-42.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 29 14:35:22.447564 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:22.447550 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 29 14:35:22.455197 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.455163 2580 manager.go:354] Recovery completed Jul 29 14:35:22.458956 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.458941 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 14:35:22.460572 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.460557 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeHasSufficientMemory" Jul 29 14:35:22.460631 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.460586 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 14:35:22.460631 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.460598 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeHasSufficientPID" Jul 29 14:35:22.461075 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.461062 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 29 14:35:22.461075 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.461069 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 14:35:22.461163 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.461073 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 29 14:35:22.461163 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.461155 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 29 14:35:22.462156 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.462142 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeHasSufficientMemory" Jul 29 14:35:22.462248 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.462168 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 14:35:22.462248 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.462179 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeHasSufficientPID" Jul 29 14:35:22.462540 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.462529 2580 policy_none.go:49] "None policy: Start" Jul 29 14:35:22.462574 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.462544 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 29 14:35:22.462574 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.462554 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 29 14:35:22.463414 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.463403 2580 policy_none.go:47] "Start" Jul 29 14:35:22.497777 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.497681 2580 manager.go:337] "Starting Device Plugin manager" Jul 29 14:35:22.497864 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:22.497793 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 29 14:35:22.497864 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.497807 2580 server.go:87] "Starting device plugin registration server" Jul 29 14:35:22.498015 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.498003 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 29 14:35:22.498073 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.498017 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 29 14:35:22.498128 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.498107 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 29 14:35:22.498231 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.498220 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 29 14:35:22.498231 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.498231 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 29 14:35:22.499955 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:22.499936 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 29 14:35:22.500044 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:22.499986 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-42.ec2.internal\" not found" Jul 29 14:35:22.598149 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.598132 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 14:35:22.599206 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.598867 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeHasSufficientMemory" Jul 29 14:35:22.599206 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.598888 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 14:35:22.599206 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.598897 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeHasSufficientPID" Jul 29 14:35:22.599206 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.598916 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-42.ec2.internal" Jul 29 14:35:22.599588 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.599562 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 29 14:35:22.600723 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.600700 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 29 14:35:22.600773 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.600746 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 29 14:35:22.600773 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.600768 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 29 14:35:22.600908 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:22.600801 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 29 14:35:22.602916 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.602898 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 14:35:22.607240 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:22.607227 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-42.ec2.internal" Jul 29 14:35:22.607305 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:22.607245 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-42.ec2.internal\": node \"ip-10-0-132-42.ec2.internal\" not found" Jul 29 14:35:22.663673 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:22.663622 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-42.ec2.internal\" not found" Jul 29 14:35:23.912293 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.911653 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-42.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-42.ec2.internal"] Jul 29 14:35:23.912354 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.912324 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 14:35:23.914973 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.914960 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeHasSufficientMemory" Jul 29 14:35:23.915038 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.914983 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 14:35:23.915038 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.914992 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeHasSufficientPID" Jul 29 14:35:23.916359 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.916346 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 14:35:23.916482 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.916467 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-42.ec2.internal" Jul 29 14:35:23.916515 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.916499 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 14:35:23.917054 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.917038 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeHasSufficientMemory" Jul 29 14:35:23.917054 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.917058 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 14:35:23.917186 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.917067 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeHasSufficientPID" Jul 29 14:35:23.917186 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.917080 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeHasSufficientMemory" Jul 29 14:35:23.917186 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.917099 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 14:35:23.917186 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.917108 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeHasSufficientPID" Jul 29 14:35:23.918027 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.918013 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-42.ec2.internal" Jul 29 14:35:23.918093 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.918035 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 14:35:23.920393 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.920379 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeHasSufficientMemory" Jul 29 14:35:23.920475 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.920402 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 14:35:23.920475 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.920411 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeHasSufficientPID" Jul 29 14:35:23.942331 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:23.942310 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-42.ec2.internal\" not found" node="ip-10-0-132-42.ec2.internal" Jul 29 14:35:23.946427 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:23.946413 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-42.ec2.internal\" not found" node="ip-10-0-132-42.ec2.internal" Jul 29 14:35:23.956602 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.956586 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ba036abba3605af2a4be113b0020ec88-config\") pod \"kube-apiserver-proxy-ip-10-0-132-42.ec2.internal\" (UID: \"ba036abba3605af2a4be113b0020ec88\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-42.ec2.internal" Jul 29 14:35:23.956677 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.956612 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4986a82ee8d2a285c70881170330a296-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-42.ec2.internal\" (UID: \"4986a82ee8d2a285c70881170330a296\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-42.ec2.internal" Jul 29 14:35:23.956677 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:23.956639 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4986a82ee8d2a285c70881170330a296-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-42.ec2.internal\" (UID: \"4986a82ee8d2a285c70881170330a296\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-42.ec2.internal" Jul 29 14:35:23.975091 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:23.975072 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-42.ec2.internal\" not found" Jul 29 14:35:24.057283 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:24.057261 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4986a82ee8d2a285c70881170330a296-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-42.ec2.internal\" (UID: \"4986a82ee8d2a285c70881170330a296\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-42.ec2.internal" Jul 29 14:35:24.057375 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:24.057286 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4986a82ee8d2a285c70881170330a296-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-42.ec2.internal\" (UID: \"4986a82ee8d2a285c70881170330a296\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-42.ec2.internal" Jul 29 14:35:24.057375 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:24.057302 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ba036abba3605af2a4be113b0020ec88-config\") pod \"kube-apiserver-proxy-ip-10-0-132-42.ec2.internal\" (UID: \"ba036abba3605af2a4be113b0020ec88\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-42.ec2.internal" Jul 29 14:35:24.057375 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:24.057347 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ba036abba3605af2a4be113b0020ec88-config\") pod \"kube-apiserver-proxy-ip-10-0-132-42.ec2.internal\" (UID: \"ba036abba3605af2a4be113b0020ec88\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-42.ec2.internal" Jul 29 14:35:24.057375 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:24.057360 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4986a82ee8d2a285c70881170330a296-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-42.ec2.internal\" (UID: \"4986a82ee8d2a285c70881170330a296\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-42.ec2.internal" Jul 29 14:35:24.057375 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:24.057373 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4986a82ee8d2a285c70881170330a296-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-42.ec2.internal\" (UID: \"4986a82ee8d2a285c70881170330a296\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-42.ec2.internal" Jul 29 14:35:24.075848 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:24.075828 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-42.ec2.internal\" not found" Jul 29 14:35:24.176372 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:24.176317 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-42.ec2.internal\" not found" Jul 29 14:35:24.246530 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:24.246500 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-42.ec2.internal" Jul 29 14:35:24.250189 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:24.250125 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-42.ec2.internal" Jul 29 14:35:24.277388 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:24.277362 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-42.ec2.internal\" not found" Jul 29 14:35:24.377944 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:24.377917 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-42.ec2.internal\" not found" Jul 29 14:35:24.478398 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:24.478335 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-42.ec2.internal\" not found" Jul 29 14:35:24.578892 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:24.578867 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-42.ec2.internal\" not found" Jul 29 14:35:24.595035 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:24.595013 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 29 14:35:24.595157 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:24.595138 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 29 14:35:24.660870 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:24.660819 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 14:30:23 +0000 UTC" deadline="2028-02-08 02:53:53.495921685 +0000 UTC" Jul 29 14:35:24.660870 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:24.660864 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13404h18m28.835061897s" Jul 29 14:35:24.676072 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:24.676050 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 14:35:24.752490 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:24.752468 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-42.ec2.internal" Jul 29 14:35:24.763325 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:24.763308 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 29 14:35:24.765111 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:24.765097 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-42.ec2.internal" Jul 29 14:35:24.773134 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:24.773118 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 29 14:35:24.869251 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:24.869226 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 14:35:24.929670 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:24.929646 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba036abba3605af2a4be113b0020ec88.slice/crio-361cbb0dbcfbed29e2107e5e7afe36eea867d7f3c9c45cff9dc80529825ee8b1 WatchSource:0}: Error finding container 361cbb0dbcfbed29e2107e5e7afe36eea867d7f3c9c45cff9dc80529825ee8b1: Status 404 returned error can't find the container with id 361cbb0dbcfbed29e2107e5e7afe36eea867d7f3c9c45cff9dc80529825ee8b1 Jul 29 14:35:24.933786 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:24.933772 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 14:35:25.001147 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.001126 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 14:35:25.494879 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.494851 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 14:35:25.634374 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.634355 2580 apiserver.go:52] "Watching apiserver" Jul 29 14:35:25.640066 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.640040 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 29 14:35:25.640511 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.640486 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4","openshift-image-registry/node-ca-gxzdh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-42.ec2.internal","openshift-network-operator/iptables-alerter-qmdq4","openshift-ovn-kubernetes/ovnkube-node-67v2n","kube-system/konnectivity-agent-99w5n","kube-system/kube-apiserver-proxy-ip-10-0-132-42.ec2.internal","openshift-cluster-node-tuning-operator/tuned-9dcks","openshift-dns/node-resolver-krcjx","openshift-multus/multus-additional-cni-plugins-n84x4","openshift-multus/multus-jrs97","openshift-multus/network-metrics-daemon-fmd7q","openshift-network-diagnostics/network-check-target-vv7bl","kube-system/global-pull-secret-syncer-tfbhf"] Jul 29 14:35:25.642592 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.642575 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.643622 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.643599 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:25.643714 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:25.643675 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:35:25.644607 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.644587 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 29 14:35:25.644857 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.644779 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tqtvj\"" Jul 29 14:35:25.644857 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.644818 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.644857 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.644827 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 29 14:35:25.644857 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.644846 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 29 14:35:25.645368 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.644808 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 29 14:35:25.645479 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.645416 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 29 14:35:25.646757 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.646249 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qmdq4" Jul 29 14:35:25.648987 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.648954 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pnkp7\"" Jul 29 14:35:25.649082 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.649021 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.649779 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.649749 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 29 14:35:25.649881 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.649754 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 29 14:35:25.649881 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.649759 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4q7j7\"" Jul 29 14:35:25.649979 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.649944 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 29 14:35:25.650178 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.650153 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 29 14:35:25.650445 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.650428 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-99w5n" Jul 29 14:35:25.650504 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.650445 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 29 14:35:25.651049 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.650918 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 29 14:35:25.651049 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.650942 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 29 14:35:25.651049 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.651008 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 29 14:35:25.651258 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.651202 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 29 14:35:25.651411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.651389 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 29 14:35:25.651594 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.651567 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gx9l8\"" Jul 29 14:35:25.652019 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.651999 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 29 14:35:25.652107 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.652033 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-krcjx" Jul 29 14:35:25.652484 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.652466 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 29 14:35:25.652615 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.652595 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6wppp\"" Jul 29 14:35:25.652683 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.652623 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 29 14:35:25.653057 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.653033 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 29 14:35:25.653443 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.653429 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:25.653515 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:25.653496 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:35:25.655710 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.655688 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 29 14:35:25.655826 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.655774 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 29 14:35:25.655826 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.655801 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gxzdh" Jul 29 14:35:25.655938 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.655804 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-fv8lx\"" Jul 29 14:35:25.657123 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.657102 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.657958 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.657933 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-j7sdl\"" Jul 29 14:35:25.658122 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.658098 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 29 14:35:25.658332 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.658315 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 29 14:35:25.658413 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.658320 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.658554 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.658539 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 29 14:35:25.659052 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.659030 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 29 14:35:25.659566 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.659549 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:25.659619 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:25.659604 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:35:25.661868 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.661842 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 29 14:35:25.661987 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.661968 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 29 14:35:25.662052 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.661989 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 29 14:35:25.662127 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.662108 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-blp8h\"" Jul 29 14:35:25.662316 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.662299 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-56l7r\"" Jul 29 14:35:25.666042 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666017 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-run\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.666042 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666040 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-host\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.666201 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666057 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5cdab631-5539-4ea1-845f-70a030e2c6b2-etc-tuned\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.666201 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666084 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-os-release\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.666201 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666124 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-cni-binary-copy\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.666201 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666198 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j8pp\" (UniqueName: \"kubernetes.io/projected/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-kube-api-access-4j8pp\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.666374 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666241 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-run-systemd\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.666374 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666261 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/36c6b7d1-eca5-4c89-a812-ea83e6ba33a7-konnectivity-ca\") pod \"konnectivity-agent-99w5n\" (UID: \"36c6b7d1-eca5-4c89-a812-ea83e6ba33a7\") " pod="kube-system/konnectivity-agent-99w5n" Jul 29 14:35:25.666374 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666276 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs\") pod \"network-metrics-daemon-fmd7q\" (UID: \"1e27a282-f2d3-4aa9-81d3-468c1da0ab8e\") " pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:25.666374 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666323 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-etc-systemd\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.666374 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666354 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-sys\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.666374 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666375 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-host-kubelet\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.666575 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666390 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/64bb03c8-45a0-470b-a207-e1e660b6faf2-env-overrides\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.666575 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666404 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk8j7\" (UniqueName: \"kubernetes.io/projected/5cdab631-5539-4ea1-845f-70a030e2c6b2-kube-api-access-qk8j7\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.666575 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666440 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d695c7f-6233-4e98-a4f1-1a35f31ac624-host-slash\") pod \"iptables-alerter-qmdq4\" (UID: \"5d695c7f-6233-4e98-a4f1-1a35f31ac624\") " pod="openshift-network-operator/iptables-alerter-qmdq4" Jul 29 14:35:25.666575 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666476 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xlqd\" (UniqueName: \"kubernetes.io/projected/5d695c7f-6233-4e98-a4f1-1a35f31ac624-kube-api-access-9xlqd\") pod \"iptables-alerter-qmdq4\" (UID: \"5d695c7f-6233-4e98-a4f1-1a35f31ac624\") " pod="openshift-network-operator/iptables-alerter-qmdq4" Jul 29 14:35:25.666575 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666506 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a1e60e32-5fbd-46a4-9458-fab72b4b4b68-hosts-file\") pod \"node-resolver-krcjx\" (UID: \"a1e60e32-5fbd-46a4-9458-fab72b4b4b68\") " pod="openshift-dns/node-resolver-krcjx" Jul 29 14:35:25.666575 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666529 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1e60e32-5fbd-46a4-9458-fab72b4b4b68-tmp-dir\") pod \"node-resolver-krcjx\" (UID: \"a1e60e32-5fbd-46a4-9458-fab72b4b4b68\") " pod="openshift-dns/node-resolver-krcjx" Jul 29 14:35:25.666575 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666553 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.666575 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666576 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/64bb03c8-45a0-470b-a207-e1e660b6faf2-ovn-node-metrics-cert\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.666846 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666601 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/36c6b7d1-eca5-4c89-a812-ea83e6ba33a7-agent-certs\") pod \"konnectivity-agent-99w5n\" (UID: \"36c6b7d1-eca5-4c89-a812-ea83e6ba33a7\") " pod="kube-system/konnectivity-agent-99w5n" Jul 29 14:35:25.666846 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666625 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-var-lib-kubelet\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.666846 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666646 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5cdab631-5539-4ea1-845f-70a030e2c6b2-tmp\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.666846 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666670 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-system-cni-dir\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.666846 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666696 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.666846 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666738 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-run-openvswitch\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.666846 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666768 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-host-cni-bin\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.666846 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666814 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-etc-openvswitch\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.666846 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666842 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.667138 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666869 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/64bb03c8-45a0-470b-a207-e1e660b6faf2-ovnkube-config\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.667138 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666897 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/64bb03c8-45a0-470b-a207-e1e660b6faf2-ovnkube-script-lib\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.667138 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666921 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-etc-modprobe-d\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.667138 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666943 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-etc-sysconfig\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.667138 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666974 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb6d8\" (UniqueName: \"kubernetes.io/projected/a1e60e32-5fbd-46a4-9458-fab72b4b4b68-kube-api-access-mb6d8\") pod \"node-resolver-krcjx\" (UID: \"a1e60e32-5fbd-46a4-9458-fab72b4b4b68\") " pod="openshift-dns/node-resolver-krcjx" Jul 29 14:35:25.667138 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.666996 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zx62\" (UniqueName: \"kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62\") pod \"network-check-target-vv7bl\" (UID: \"10c2f512-d00b-4f28-8957-4311e81257ec\") " pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:25.667138 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.667016 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-systemd-units\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.667138 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.667053 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-run-ovn\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.667138 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.667067 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-etc-sysctl-d\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.667138 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.667081 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-lib-modules\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.667138 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.667116 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5d695c7f-6233-4e98-a4f1-1a35f31ac624-iptables-alerter-script\") pod \"iptables-alerter-qmdq4\" (UID: \"5d695c7f-6233-4e98-a4f1-1a35f31ac624\") " pod="openshift-network-operator/iptables-alerter-qmdq4" Jul 29 14:35:25.667138 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.667138 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-host-slash\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.667482 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.667152 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-host-run-netns\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.667482 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.667189 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-var-lib-openvswitch\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.667482 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.667204 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-log-socket\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.667482 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.667219 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-host-run-ovn-kubernetes\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.667482 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.667238 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-host-cni-netd\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.667482 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.667252 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pbdq\" (UniqueName: \"kubernetes.io/projected/64bb03c8-45a0-470b-a207-e1e660b6faf2-kube-api-access-2pbdq\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.667482 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.667280 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-etc-kubernetes\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.667482 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.667305 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-cnibin\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.667482 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.667325 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.667482 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.667344 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-node-log\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.667482 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.667361 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hq74\" (UniqueName: \"kubernetes.io/projected/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-kube-api-access-7hq74\") pod \"network-metrics-daemon-fmd7q\" (UID: \"1e27a282-f2d3-4aa9-81d3-468c1da0ab8e\") " pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:25.667482 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.667382 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-etc-sysctl-conf\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.667831 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.667787 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 29 14:35:25.696667 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.696647 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gzjxj" Jul 29 14:35:25.702230 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.702211 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gzjxj" Jul 29 14:35:25.753646 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.753599 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 29 14:35:25.768245 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768222 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vnmt\" (UniqueName: \"kubernetes.io/projected/0af98f75-498c-4059-924a-daa3ab0ebfbc-kube-api-access-5vnmt\") pod \"node-ca-gxzdh\" (UID: \"0af98f75-498c-4059-924a-daa3ab0ebfbc\") " pod="openshift-image-registry/node-ca-gxzdh" Jul 29 14:35:25.768339 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768248 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-systemd-units\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.768339 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768271 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-run-ovn\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.768339 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768286 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-etc-sysctl-d\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.768339 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768304 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-host-var-lib-kubelet\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.768339 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768318 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret\") pod \"global-pull-secret-syncer-tfbhf\" (UID: \"3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9\") " pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:25.768339 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768333 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drxrc\" (UniqueName: \"kubernetes.io/projected/6b1c56e1-11a4-451e-9982-f4b47335ff9f-kube-api-access-drxrc\") pod \"aws-ebs-csi-driver-node-ssdb4\" (UID: \"6b1c56e1-11a4-451e-9982-f4b47335ff9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.768611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768347 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-host-run-netns\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.768611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768362 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-var-lib-openvswitch\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.768611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768375 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-log-socket\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.768611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768391 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-host-run-ovn-kubernetes\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.768611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768346 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-systemd-units\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.768611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768414 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-run-ovn\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.768611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768408 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-host-cni-netd\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.768611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768434 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-etc-sysctl-d\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.768611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768464 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-host\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.768611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768474 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-host-cni-netd\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.768611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768489 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk8j7\" (UniqueName: \"kubernetes.io/projected/5cdab631-5539-4ea1-845f-70a030e2c6b2-kube-api-access-qk8j7\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.768611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768477 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-host-run-ovn-kubernetes\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.768611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768516 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-cnibin\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.768611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768529 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-host\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.768611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768532 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-var-lib-openvswitch\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.768611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768540 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-cnibin\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.768611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768553 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-host-run-netns\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.769403 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768572 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.769403 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768583 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-cnibin\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.769403 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768600 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hq74\" (UniqueName: \"kubernetes.io/projected/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-kube-api-access-7hq74\") pod \"network-metrics-daemon-fmd7q\" (UID: \"1e27a282-f2d3-4aa9-81d3-468c1da0ab8e\") " pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:25.769403 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768623 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5cdab631-5539-4ea1-845f-70a030e2c6b2-etc-tuned\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.769403 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768626 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-log-socket\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.769403 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768647 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-multus-daemon-config\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.769403 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768671 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m689k\" (UniqueName: \"kubernetes.io/projected/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-kube-api-access-m689k\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.769403 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768694 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b1c56e1-11a4-451e-9982-f4b47335ff9f-socket-dir\") pod \"aws-ebs-csi-driver-node-ssdb4\" (UID: \"6b1c56e1-11a4-451e-9982-f4b47335ff9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.769403 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768717 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b1c56e1-11a4-451e-9982-f4b47335ff9f-etc-selinux\") pod \"aws-ebs-csi-driver-node-ssdb4\" (UID: \"6b1c56e1-11a4-451e-9982-f4b47335ff9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.769403 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768776 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/36c6b7d1-eca5-4c89-a812-ea83e6ba33a7-konnectivity-ca\") pod \"konnectivity-agent-99w5n\" (UID: \"36c6b7d1-eca5-4c89-a812-ea83e6ba33a7\") " pod="kube-system/konnectivity-agent-99w5n" Jul 29 14:35:25.769403 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768804 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs\") pod \"network-metrics-daemon-fmd7q\" (UID: \"1e27a282-f2d3-4aa9-81d3-468c1da0ab8e\") " pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:25.769403 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768826 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-sys\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.769403 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768848 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d695c7f-6233-4e98-a4f1-1a35f31ac624-host-slash\") pod \"iptables-alerter-qmdq4\" (UID: \"5d695c7f-6233-4e98-a4f1-1a35f31ac624\") " pod="openshift-network-operator/iptables-alerter-qmdq4" Jul 29 14:35:25.769403 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768871 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-multus-cni-dir\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.769403 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768894 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/64bb03c8-45a0-470b-a207-e1e660b6faf2-env-overrides\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.769403 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768903 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-sys\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.769403 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768913 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d695c7f-6233-4e98-a4f1-1a35f31ac624-host-slash\") pod \"iptables-alerter-qmdq4\" (UID: \"5d695c7f-6233-4e98-a4f1-1a35f31ac624\") " pod="openshift-network-operator/iptables-alerter-qmdq4" Jul 29 14:35:25.770193 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768914 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 29 14:35:25.770193 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768915 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-kubelet-config\") pod \"global-pull-secret-syncer-tfbhf\" (UID: \"3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9\") " pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:25.770193 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768959 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0af98f75-498c-4059-924a-daa3ab0ebfbc-host\") pod \"node-ca-gxzdh\" (UID: \"0af98f75-498c-4059-924a-daa3ab0ebfbc\") " pod="openshift-image-registry/node-ca-gxzdh" Jul 29 14:35:25.770193 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:25.768971 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 14:35:25.770193 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.768982 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b1c56e1-11a4-451e-9982-f4b47335ff9f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ssdb4\" (UID: \"6b1c56e1-11a4-451e-9982-f4b47335ff9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.770193 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769010 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/64bb03c8-45a0-470b-a207-e1e660b6faf2-ovn-node-metrics-cert\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.770193 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:25.769035 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs podName:1e27a282-f2d3-4aa9-81d3-468c1da0ab8e nodeName:}" failed. No retries permitted until 2026-07-29 14:35:26.269002794 +0000 UTC m=+2.997151961 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs") pod "network-metrics-daemon-fmd7q" (UID: "1e27a282-f2d3-4aa9-81d3-468c1da0ab8e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 14:35:25.770193 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769069 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-var-lib-kubelet\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.770193 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769100 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-multus-conf-dir\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.770193 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769126 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-system-cni-dir\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.770193 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769164 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.770193 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769204 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-var-lib-kubelet\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.770193 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769168 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-system-cni-dir\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.770193 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769337 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/64bb03c8-45a0-470b-a207-e1e660b6faf2-env-overrides\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.770193 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769337 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/36c6b7d1-eca5-4c89-a812-ea83e6ba33a7-konnectivity-ca\") pod \"konnectivity-agent-99w5n\" (UID: \"36c6b7d1-eca5-4c89-a812-ea83e6ba33a7\") " pod="kube-system/konnectivity-agent-99w5n" Jul 29 14:35:25.770193 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769439 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.770193 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769476 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-host-cni-bin\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.770991 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-os-release\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.770991 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769535 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-host-cni-bin\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.770991 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769537 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-host-run-k8s-cni-cncf-io\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.770991 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769579 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-host-run-multus-certs\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.770991 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769599 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-dbus\") pod \"global-pull-secret-syncer-tfbhf\" (UID: \"3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9\") " pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:25.770991 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769621 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-etc-openvswitch\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.770991 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769642 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/64bb03c8-45a0-470b-a207-e1e660b6faf2-ovnkube-script-lib\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.770991 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769661 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-etc-sysconfig\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.770991 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769681 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-host-var-lib-cni-bin\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.770991 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769702 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb6d8\" (UniqueName: \"kubernetes.io/projected/a1e60e32-5fbd-46a4-9458-fab72b4b4b68-kube-api-access-mb6d8\") pod \"node-resolver-krcjx\" (UID: \"a1e60e32-5fbd-46a4-9458-fab72b4b4b68\") " pod="openshift-dns/node-resolver-krcjx" Jul 29 14:35:25.770991 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769741 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-etc-sysconfig\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.770991 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769739 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zx62\" (UniqueName: \"kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62\") pod \"network-check-target-vv7bl\" (UID: \"10c2f512-d00b-4f28-8957-4311e81257ec\") " pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:25.770991 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769795 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-lib-modules\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.770991 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769834 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-system-cni-dir\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.770991 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769861 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-host-slash\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.770991 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769885 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pbdq\" (UniqueName: \"kubernetes.io/projected/64bb03c8-45a0-470b-a207-e1e660b6faf2-kube-api-access-2pbdq\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.770991 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769910 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-etc-kubernetes\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.771844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769932 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-etc-sysctl-conf\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.771844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769974 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-host-slash\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.771844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769982 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.771844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770029 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-run\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.771844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770041 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-etc-sysctl-conf\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.771844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770056 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-hostroot\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.771844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770080 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-lib-modules\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.771844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770098 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-etc-kubernetes\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.771844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770114 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5cdab631-5539-4ea1-845f-70a030e2c6b2-tmp\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.771844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770130 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/64bb03c8-45a0-470b-a207-e1e660b6faf2-ovnkube-script-lib\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.771844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770144 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-run\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.771844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770146 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5d695c7f-6233-4e98-a4f1-1a35f31ac624-iptables-alerter-script\") pod \"iptables-alerter-qmdq4\" (UID: \"5d695c7f-6233-4e98-a4f1-1a35f31ac624\") " pod="openshift-network-operator/iptables-alerter-qmdq4" Jul 29 14:35:25.771844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770176 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-node-log\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.771844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770198 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-os-release\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.771844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770221 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-cni-binary-copy\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.771844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770245 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j8pp\" (UniqueName: \"kubernetes.io/projected/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-kube-api-access-4j8pp\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.771844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.769716 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-etc-openvswitch\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.772613 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770270 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-run-systemd\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.772613 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770292 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-etc-systemd\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.772613 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770299 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-os-release\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.772613 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770319 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xlqd\" (UniqueName: \"kubernetes.io/projected/5d695c7f-6233-4e98-a4f1-1a35f31ac624-kube-api-access-9xlqd\") pod \"iptables-alerter-qmdq4\" (UID: \"5d695c7f-6233-4e98-a4f1-1a35f31ac624\") " pod="openshift-network-operator/iptables-alerter-qmdq4" Jul 29 14:35:25.772613 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770341 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-node-log\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.772613 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770346 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-multus-socket-dir-parent\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.772613 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770370 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-etc-kubernetes\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.772613 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770395 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-host-kubelet\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.772613 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770396 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-run-systemd\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.772613 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770432 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b1c56e1-11a4-451e-9982-f4b47335ff9f-device-dir\") pod \"aws-ebs-csi-driver-node-ssdb4\" (UID: \"6b1c56e1-11a4-451e-9982-f4b47335ff9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.772613 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770458 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b1c56e1-11a4-451e-9982-f4b47335ff9f-sys-fs\") pod \"aws-ebs-csi-driver-node-ssdb4\" (UID: \"6b1c56e1-11a4-451e-9982-f4b47335ff9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.772613 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770462 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-host-kubelet\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.772613 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770477 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-etc-systemd\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.772613 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770501 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a1e60e32-5fbd-46a4-9458-fab72b4b4b68-hosts-file\") pod \"node-resolver-krcjx\" (UID: \"a1e60e32-5fbd-46a4-9458-fab72b4b4b68\") " pod="openshift-dns/node-resolver-krcjx" Jul 29 14:35:25.772613 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770536 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1e60e32-5fbd-46a4-9458-fab72b4b4b68-tmp-dir\") pod \"node-resolver-krcjx\" (UID: \"a1e60e32-5fbd-46a4-9458-fab72b4b4b68\") " pod="openshift-dns/node-resolver-krcjx" Jul 29 14:35:25.772613 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770545 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a1e60e32-5fbd-46a4-9458-fab72b4b4b68-hosts-file\") pod \"node-resolver-krcjx\" (UID: \"a1e60e32-5fbd-46a4-9458-fab72b4b4b68\") " pod="openshift-dns/node-resolver-krcjx" Jul 29 14:35:25.772613 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770581 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5d695c7f-6233-4e98-a4f1-1a35f31ac624-iptables-alerter-script\") pod \"iptables-alerter-qmdq4\" (UID: \"5d695c7f-6233-4e98-a4f1-1a35f31ac624\") " pod="openshift-network-operator/iptables-alerter-qmdq4" Jul 29 14:35:25.773109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770625 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.773109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770652 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/36c6b7d1-eca5-4c89-a812-ea83e6ba33a7-agent-certs\") pod \"konnectivity-agent-99w5n\" (UID: \"36c6b7d1-eca5-4c89-a812-ea83e6ba33a7\") " pod="kube-system/konnectivity-agent-99w5n" Jul 29 14:35:25.773109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770680 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-host-var-lib-cni-multus\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.773109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770746 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-run-openvswitch\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.773109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770769 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0af98f75-498c-4059-924a-daa3ab0ebfbc-serviceca\") pod \"node-ca-gxzdh\" (UID: \"0af98f75-498c-4059-924a-daa3ab0ebfbc\") " pod="openshift-image-registry/node-ca-gxzdh" Jul 29 14:35:25.773109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770791 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b1c56e1-11a4-451e-9982-f4b47335ff9f-registration-dir\") pod \"aws-ebs-csi-driver-node-ssdb4\" (UID: \"6b1c56e1-11a4-451e-9982-f4b47335ff9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.773109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770815 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.773109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770841 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/64bb03c8-45a0-470b-a207-e1e660b6faf2-ovnkube-config\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.773109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770842 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.773109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770865 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-etc-modprobe-d\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.773109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770889 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1e60e32-5fbd-46a4-9458-fab72b4b4b68-tmp-dir\") pod \"node-resolver-krcjx\" (UID: \"a1e60e32-5fbd-46a4-9458-fab72b4b4b68\") " pod="openshift-dns/node-resolver-krcjx" Jul 29 14:35:25.773109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770891 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-cni-binary-copy\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.773109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770933 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-run-openvswitch\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.773109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770953 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5cdab631-5539-4ea1-845f-70a030e2c6b2-etc-modprobe-d\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.773109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770963 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64bb03c8-45a0-470b-a207-e1e660b6faf2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.773109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.770983 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-cni-binary-copy\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.773109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.771029 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-host-run-netns\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.773546 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.771337 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/64bb03c8-45a0-470b-a207-e1e660b6faf2-ovnkube-config\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.773546 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.772250 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5cdab631-5539-4ea1-845f-70a030e2c6b2-etc-tuned\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.773546 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.772429 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/64bb03c8-45a0-470b-a207-e1e660b6faf2-ovn-node-metrics-cert\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.773546 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.772774 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5cdab631-5539-4ea1-845f-70a030e2c6b2-tmp\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.773546 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.772885 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/36c6b7d1-eca5-4c89-a812-ea83e6ba33a7-agent-certs\") pod \"konnectivity-agent-99w5n\" (UID: \"36c6b7d1-eca5-4c89-a812-ea83e6ba33a7\") " pod="kube-system/konnectivity-agent-99w5n" Jul 29 14:35:25.781099 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:25.781079 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 14:35:25.781185 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:25.781102 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 14:35:25.781185 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:25.781114 2580 projected.go:196] Error preparing data for projected volume kube-api-access-7zx62 for pod openshift-network-diagnostics/network-check-target-vv7bl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 14:35:25.781185 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:25.781162 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62 podName:10c2f512-d00b-4f28-8957-4311e81257ec nodeName:}" failed. No retries permitted until 2026-07-29 14:35:26.2811468 +0000 UTC m=+3.009296004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7zx62" (UniqueName: "kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62") pod "network-check-target-vv7bl" (UID: "10c2f512-d00b-4f28-8957-4311e81257ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 14:35:25.782289 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.782227 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hq74\" (UniqueName: \"kubernetes.io/projected/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-kube-api-access-7hq74\") pod \"network-metrics-daemon-fmd7q\" (UID: \"1e27a282-f2d3-4aa9-81d3-468c1da0ab8e\") " pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:25.782455 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.782426 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk8j7\" (UniqueName: \"kubernetes.io/projected/5cdab631-5539-4ea1-845f-70a030e2c6b2-kube-api-access-qk8j7\") pod \"tuned-9dcks\" (UID: \"5cdab631-5539-4ea1-845f-70a030e2c6b2\") " pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.783646 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.783629 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xlqd\" (UniqueName: \"kubernetes.io/projected/5d695c7f-6233-4e98-a4f1-1a35f31ac624-kube-api-access-9xlqd\") pod \"iptables-alerter-qmdq4\" (UID: \"5d695c7f-6233-4e98-a4f1-1a35f31ac624\") " pod="openshift-network-operator/iptables-alerter-qmdq4" Jul 29 14:35:25.783803 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.783783 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j8pp\" (UniqueName: \"kubernetes.io/projected/7c5930f2-ccc2-476a-8d31-9e85d2735b7a-kube-api-access-4j8pp\") pod \"multus-additional-cni-plugins-n84x4\" (UID: \"7c5930f2-ccc2-476a-8d31-9e85d2735b7a\") " pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.784138 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.784118 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pbdq\" (UniqueName: \"kubernetes.io/projected/64bb03c8-45a0-470b-a207-e1e660b6faf2-kube-api-access-2pbdq\") pod \"ovnkube-node-67v2n\" (UID: \"64bb03c8-45a0-470b-a207-e1e660b6faf2\") " pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.784220 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.784206 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb6d8\" (UniqueName: \"kubernetes.io/projected/a1e60e32-5fbd-46a4-9458-fab72b4b4b68-kube-api-access-mb6d8\") pod \"node-resolver-krcjx\" (UID: \"a1e60e32-5fbd-46a4-9458-fab72b4b4b68\") " pod="openshift-dns/node-resolver-krcjx" Jul 29 14:35:25.815688 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.815650 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-42.ec2.internal" event={"ID":"ba036abba3605af2a4be113b0020ec88","Type":"ContainerStarted","Data":"361cbb0dbcfbed29e2107e5e7afe36eea867d7f3c9c45cff9dc80529825ee8b1"} Jul 29 14:35:25.871353 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871328 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-multus-daemon-config\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.871442 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871368 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m689k\" (UniqueName: \"kubernetes.io/projected/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-kube-api-access-m689k\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.871442 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871389 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b1c56e1-11a4-451e-9982-f4b47335ff9f-socket-dir\") pod \"aws-ebs-csi-driver-node-ssdb4\" (UID: \"6b1c56e1-11a4-451e-9982-f4b47335ff9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.871442 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871404 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b1c56e1-11a4-451e-9982-f4b47335ff9f-etc-selinux\") pod \"aws-ebs-csi-driver-node-ssdb4\" (UID: \"6b1c56e1-11a4-451e-9982-f4b47335ff9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.871442 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871438 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-multus-cni-dir\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.871650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871461 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-kubelet-config\") pod \"global-pull-secret-syncer-tfbhf\" (UID: \"3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9\") " pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:25.871650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871482 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0af98f75-498c-4059-924a-daa3ab0ebfbc-host\") pod \"node-ca-gxzdh\" (UID: \"0af98f75-498c-4059-924a-daa3ab0ebfbc\") " pod="openshift-image-registry/node-ca-gxzdh" Jul 29 14:35:25.871650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871504 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b1c56e1-11a4-451e-9982-f4b47335ff9f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ssdb4\" (UID: \"6b1c56e1-11a4-451e-9982-f4b47335ff9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.871650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871511 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b1c56e1-11a4-451e-9982-f4b47335ff9f-etc-selinux\") pod \"aws-ebs-csi-driver-node-ssdb4\" (UID: \"6b1c56e1-11a4-451e-9982-f4b47335ff9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.871650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871533 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b1c56e1-11a4-451e-9982-f4b47335ff9f-socket-dir\") pod \"aws-ebs-csi-driver-node-ssdb4\" (UID: \"6b1c56e1-11a4-451e-9982-f4b47335ff9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.871650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871552 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-multus-conf-dir\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.871650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871521 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-multus-conf-dir\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.871650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871587 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-os-release\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.871650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871602 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-multus-cni-dir\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.871650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871603 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-host-run-k8s-cni-cncf-io\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.871650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871622 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-host-run-k8s-cni-cncf-io\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.871650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871630 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-host-run-multus-certs\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.871650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871645 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-dbus\") pod \"global-pull-secret-syncer-tfbhf\" (UID: \"3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9\") " pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:25.872030 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871657 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-os-release\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.872030 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871660 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-host-var-lib-cni-bin\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.872030 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871686 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-host-var-lib-cni-bin\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.872030 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871694 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-system-cni-dir\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.872030 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871712 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-hostroot\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.872030 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871714 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-kubelet-config\") pod \"global-pull-secret-syncer-tfbhf\" (UID: \"3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9\") " pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:25.872030 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871754 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-multus-socket-dir-parent\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.872030 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871758 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-hostroot\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.872030 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871770 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-etc-kubernetes\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.872030 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871786 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b1c56e1-11a4-451e-9982-f4b47335ff9f-device-dir\") pod \"aws-ebs-csi-driver-node-ssdb4\" (UID: \"6b1c56e1-11a4-451e-9982-f4b47335ff9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.872030 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871793 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-system-cni-dir\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.872030 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871800 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b1c56e1-11a4-451e-9982-f4b47335ff9f-sys-fs\") pod \"aws-ebs-csi-driver-node-ssdb4\" (UID: \"6b1c56e1-11a4-451e-9982-f4b47335ff9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.872030 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871815 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0af98f75-498c-4059-924a-daa3ab0ebfbc-host\") pod \"node-ca-gxzdh\" (UID: \"0af98f75-498c-4059-924a-daa3ab0ebfbc\") " pod="openshift-image-registry/node-ca-gxzdh" Jul 29 14:35:25.872030 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871819 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-host-var-lib-cni-multus\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.872030 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871833 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0af98f75-498c-4059-924a-daa3ab0ebfbc-serviceca\") pod \"node-ca-gxzdh\" (UID: \"0af98f75-498c-4059-924a-daa3ab0ebfbc\") " pod="openshift-image-registry/node-ca-gxzdh" Jul 29 14:35:25.872030 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871837 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b1c56e1-11a4-451e-9982-f4b47335ff9f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ssdb4\" (UID: \"6b1c56e1-11a4-451e-9982-f4b47335ff9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.872030 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871847 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b1c56e1-11a4-451e-9982-f4b47335ff9f-registration-dir\") pod \"aws-ebs-csi-driver-node-ssdb4\" (UID: \"6b1c56e1-11a4-451e-9982-f4b47335ff9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.872030 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871872 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-dbus\") pod \"global-pull-secret-syncer-tfbhf\" (UID: \"3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9\") " pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:25.872890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871879 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b1c56e1-11a4-451e-9982-f4b47335ff9f-registration-dir\") pod \"aws-ebs-csi-driver-node-ssdb4\" (UID: \"6b1c56e1-11a4-451e-9982-f4b47335ff9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.872890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871882 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-cni-binary-copy\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.872890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871898 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-multus-socket-dir-parent\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.872890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871799 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-host-run-multus-certs\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.872890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871906 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-host-run-netns\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.872890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871911 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-multus-daemon-config\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.872890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871928 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-host-var-lib-cni-multus\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.872890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871930 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vnmt\" (UniqueName: \"kubernetes.io/projected/0af98f75-498c-4059-924a-daa3ab0ebfbc-kube-api-access-5vnmt\") pod \"node-ca-gxzdh\" (UID: \"0af98f75-498c-4059-924a-daa3ab0ebfbc\") " pod="openshift-image-registry/node-ca-gxzdh" Jul 29 14:35:25.872890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871942 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b1c56e1-11a4-451e-9982-f4b47335ff9f-sys-fs\") pod \"aws-ebs-csi-driver-node-ssdb4\" (UID: \"6b1c56e1-11a4-451e-9982-f4b47335ff9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.872890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871841 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-etc-kubernetes\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.872890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871960 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-host-var-lib-kubelet\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.872890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871976 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret\") pod \"global-pull-secret-syncer-tfbhf\" (UID: \"3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9\") " pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:25.872890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871978 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-host-run-netns\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.872890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871990 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drxrc\" (UniqueName: \"kubernetes.io/projected/6b1c56e1-11a4-451e-9982-f4b47335ff9f-kube-api-access-drxrc\") pod \"aws-ebs-csi-driver-node-ssdb4\" (UID: \"6b1c56e1-11a4-451e-9982-f4b47335ff9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.872890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.872014 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-cnibin\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.872890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.871908 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b1c56e1-11a4-451e-9982-f4b47335ff9f-device-dir\") pod \"aws-ebs-csi-driver-node-ssdb4\" (UID: \"6b1c56e1-11a4-451e-9982-f4b47335ff9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.872890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.872106 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-cnibin\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.872890 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:25.872168 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 14:35:25.873691 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:25.872224 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret podName:3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9 nodeName:}" failed. No retries permitted until 2026-07-29 14:35:26.372209315 +0000 UTC m=+3.100358497 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret") pod "global-pull-secret-syncer-tfbhf" (UID: "3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9") : object "kube-system"/"original-pull-secret" not registered Jul 29 14:35:25.873691 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.872254 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-host-var-lib-kubelet\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.873691 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.872337 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-cni-binary-copy\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.873691 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.873024 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/0af98f75-498c-4059-924a-daa3ab0ebfbc-serviceca\") pod \"node-ca-gxzdh\" (UID: \"0af98f75-498c-4059-924a-daa3ab0ebfbc\") " pod="openshift-image-registry/node-ca-gxzdh" Jul 29 14:35:25.879322 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.879300 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vnmt\" (UniqueName: \"kubernetes.io/projected/0af98f75-498c-4059-924a-daa3ab0ebfbc-kube-api-access-5vnmt\") pod \"node-ca-gxzdh\" (UID: \"0af98f75-498c-4059-924a-daa3ab0ebfbc\") " pod="openshift-image-registry/node-ca-gxzdh" Jul 29 14:35:25.882739 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.882675 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drxrc\" (UniqueName: \"kubernetes.io/projected/6b1c56e1-11a4-451e-9982-f4b47335ff9f-kube-api-access-drxrc\") pod \"aws-ebs-csi-driver-node-ssdb4\" (UID: \"6b1c56e1-11a4-451e-9982-f4b47335ff9f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.882835 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.882817 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m689k\" (UniqueName: \"kubernetes.io/projected/a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d-kube-api-access-m689k\") pod \"multus-jrs97\" (UID: \"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d\") " pod="openshift-multus/multus-jrs97" Jul 29 14:35:25.923751 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:25.923718 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4986a82ee8d2a285c70881170330a296.slice/crio-bc39ac513faf3d997eafcf934c48395e868bf4be5572e1636e83653bfda8f05b WatchSource:0}: Error finding container bc39ac513faf3d997eafcf934c48395e868bf4be5572e1636e83653bfda8f05b: Status 404 returned error can't find the container with id bc39ac513faf3d997eafcf934c48395e868bf4be5572e1636e83653bfda8f05b Jul 29 14:35:25.954269 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.954249 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n84x4" Jul 29 14:35:25.961050 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.961031 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9dcks" Jul 29 14:35:25.963070 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:25.962990 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c5930f2_ccc2_476a_8d31_9e85d2735b7a.slice/crio-0463f36df7d8d06314d794d08d0ee94beb535c9a18a6fc08df2c21551ffa317e WatchSource:0}: Error finding container 0463f36df7d8d06314d794d08d0ee94beb535c9a18a6fc08df2c21551ffa317e: Status 404 returned error can't find the container with id 0463f36df7d8d06314d794d08d0ee94beb535c9a18a6fc08df2c21551ffa317e Jul 29 14:35:25.968448 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.968428 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qmdq4" Jul 29 14:35:25.968697 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:25.968675 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cdab631_5539_4ea1_845f_70a030e2c6b2.slice/crio-028d785047c22d40030f95bcfc35b93b8f5ddbc639b4f310ecabaad650a3b1d4 WatchSource:0}: Error finding container 028d785047c22d40030f95bcfc35b93b8f5ddbc639b4f310ecabaad650a3b1d4: Status 404 returned error can't find the container with id 028d785047c22d40030f95bcfc35b93b8f5ddbc639b4f310ecabaad650a3b1d4 Jul 29 14:35:25.973773 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.973754 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:25.977467 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:25.977448 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d695c7f_6233_4e98_a4f1_1a35f31ac624.slice/crio-71c086eb7e616eeada1be95fe72241d2d0189e6bc9699d5f994ff4c0f5f3a6fb WatchSource:0}: Error finding container 71c086eb7e616eeada1be95fe72241d2d0189e6bc9699d5f994ff4c0f5f3a6fb: Status 404 returned error can't find the container with id 71c086eb7e616eeada1be95fe72241d2d0189e6bc9699d5f994ff4c0f5f3a6fb Jul 29 14:35:25.978294 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.978272 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-99w5n" Jul 29 14:35:25.982453 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:25.982431 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64bb03c8_45a0_470b_a207_e1e660b6faf2.slice/crio-3aecb3f40c7b8b0b9cc81f6c04276a49a8a79c0c7c0f7822043311276f36a73c WatchSource:0}: Error finding container 3aecb3f40c7b8b0b9cc81f6c04276a49a8a79c0c7c0f7822043311276f36a73c: Status 404 returned error can't find the container with id 3aecb3f40c7b8b0b9cc81f6c04276a49a8a79c0c7c0f7822043311276f36a73c Jul 29 14:35:25.983658 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.983638 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-krcjx" Jul 29 14:35:25.987246 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:25.987226 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36c6b7d1_eca5_4c89_a812_ea83e6ba33a7.slice/crio-a3d2f75914abb9d164920b53ef869db95e5ec5260a807dd2dc18bf8bca0d967e WatchSource:0}: Error finding container a3d2f75914abb9d164920b53ef869db95e5ec5260a807dd2dc18bf8bca0d967e: Status 404 returned error can't find the container with id a3d2f75914abb9d164920b53ef869db95e5ec5260a807dd2dc18bf8bca0d967e Jul 29 14:35:25.990334 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.990310 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gxzdh" Jul 29 14:35:25.991846 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:25.991828 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1e60e32_5fbd_46a4_9458_fab72b4b4b68.slice/crio-04012191e147398823ff9e744d74ae2d85106abcf386e27227677be313412540 WatchSource:0}: Error finding container 04012191e147398823ff9e744d74ae2d85106abcf386e27227677be313412540: Status 404 returned error can't find the container with id 04012191e147398823ff9e744d74ae2d85106abcf386e27227677be313412540 Jul 29 14:35:25.996498 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:25.996483 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" Jul 29 14:35:25.999073 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:25.999047 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0af98f75_498c_4059_924a_daa3ab0ebfbc.slice/crio-8571feb6bc3f646516f2be8a9cef78baca7808132aed7111c4975aa5427ba254 WatchSource:0}: Error finding container 8571feb6bc3f646516f2be8a9cef78baca7808132aed7111c4975aa5427ba254: Status 404 returned error can't find the container with id 8571feb6bc3f646516f2be8a9cef78baca7808132aed7111c4975aa5427ba254 Jul 29 14:35:26.001213 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:26.001178 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jrs97" Jul 29 14:35:26.005565 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:26.005546 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b1c56e1_11a4_451e_9982_f4b47335ff9f.slice/crio-4a768dacbbbbceee5ae47f8a2820235a653f2c31a8fabba9de39a7b58029ba0b WatchSource:0}: Error finding container 4a768dacbbbbceee5ae47f8a2820235a653f2c31a8fabba9de39a7b58029ba0b: Status 404 returned error can't find the container with id 4a768dacbbbbceee5ae47f8a2820235a653f2c31a8fabba9de39a7b58029ba0b Jul 29 14:35:26.010554 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:35:26.010535 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda609730b_db3d_4c8e_ae4a_c4b0e1eeb57d.slice/crio-4f733d3aff29a12a5608ec8b1b82cb69a508b021baa7e8c02d3392a3fadd916b WatchSource:0}: Error finding container 4f733d3aff29a12a5608ec8b1b82cb69a508b021baa7e8c02d3392a3fadd916b: Status 404 returned error can't find the container with id 4f733d3aff29a12a5608ec8b1b82cb69a508b021baa7e8c02d3392a3fadd916b Jul 29 14:35:26.275251 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:26.275199 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs\") pod \"network-metrics-daemon-fmd7q\" (UID: \"1e27a282-f2d3-4aa9-81d3-468c1da0ab8e\") " pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:26.275337 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:26.275315 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 14:35:26.275380 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:26.275363 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs podName:1e27a282-f2d3-4aa9-81d3-468c1da0ab8e nodeName:}" failed. No retries permitted until 2026-07-29 14:35:27.275349961 +0000 UTC m=+4.003499129 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs") pod "network-metrics-daemon-fmd7q" (UID: "1e27a282-f2d3-4aa9-81d3-468c1da0ab8e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 14:35:26.375950 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:26.375931 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret\") pod \"global-pull-secret-syncer-tfbhf\" (UID: \"3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9\") " pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:26.376055 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:26.375985 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zx62\" (UniqueName: \"kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62\") pod \"network-check-target-vv7bl\" (UID: \"10c2f512-d00b-4f28-8957-4311e81257ec\") " pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:26.376113 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:26.376077 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 14:35:26.376113 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:26.376080 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 14:35:26.376113 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:26.376096 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 14:35:26.376113 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:26.376108 2580 projected.go:196] Error preparing data for projected volume kube-api-access-7zx62 for pod openshift-network-diagnostics/network-check-target-vv7bl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 14:35:26.376308 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:26.376146 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret podName:3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9 nodeName:}" failed. No retries permitted until 2026-07-29 14:35:27.376126254 +0000 UTC m=+4.104275441 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret") pod "global-pull-secret-syncer-tfbhf" (UID: "3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9") : object "kube-system"/"original-pull-secret" not registered Jul 29 14:35:26.376308 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:26.376167 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62 podName:10c2f512-d00b-4f28-8957-4311e81257ec nodeName:}" failed. No retries permitted until 2026-07-29 14:35:27.376157069 +0000 UTC m=+4.104306241 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7zx62" (UniqueName: "kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62") pod "network-check-target-vv7bl" (UID: "10c2f512-d00b-4f28-8957-4311e81257ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 14:35:26.704659 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:26.704529 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 14:30:25 +0000 UTC" deadline="2028-02-18 15:00:28.153436327 +0000 UTC" Jul 29 14:35:26.704659 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:26.704567 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13656h25m1.44887481s" Jul 29 14:35:26.821117 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:26.821079 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-krcjx" event={"ID":"a1e60e32-5fbd-46a4-9458-fab72b4b4b68","Type":"ContainerStarted","Data":"04012191e147398823ff9e744d74ae2d85106abcf386e27227677be313412540"} Jul 29 14:35:26.822686 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:26.822620 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-99w5n" event={"ID":"36c6b7d1-eca5-4c89-a812-ea83e6ba33a7","Type":"ContainerStarted","Data":"a3d2f75914abb9d164920b53ef869db95e5ec5260a807dd2dc18bf8bca0d967e"} Jul 29 14:35:26.824499 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:26.824423 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qmdq4" event={"ID":"5d695c7f-6233-4e98-a4f1-1a35f31ac624","Type":"ContainerStarted","Data":"71c086eb7e616eeada1be95fe72241d2d0189e6bc9699d5f994ff4c0f5f3a6fb"} Jul 29 14:35:26.827599 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:26.827546 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9dcks" event={"ID":"5cdab631-5539-4ea1-845f-70a030e2c6b2","Type":"ContainerStarted","Data":"028d785047c22d40030f95bcfc35b93b8f5ddbc639b4f310ecabaad650a3b1d4"} Jul 29 14:35:26.830222 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:26.830159 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-42.ec2.internal" event={"ID":"4986a82ee8d2a285c70881170330a296","Type":"ContainerStarted","Data":"bc39ac513faf3d997eafcf934c48395e868bf4be5572e1636e83653bfda8f05b"} Jul 29 14:35:26.832385 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:26.832318 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jrs97" event={"ID":"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d","Type":"ContainerStarted","Data":"4f733d3aff29a12a5608ec8b1b82cb69a508b021baa7e8c02d3392a3fadd916b"} Jul 29 14:35:26.834053 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:26.833984 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" event={"ID":"6b1c56e1-11a4-451e-9982-f4b47335ff9f","Type":"ContainerStarted","Data":"4a768dacbbbbceee5ae47f8a2820235a653f2c31a8fabba9de39a7b58029ba0b"} Jul 29 14:35:26.837530 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:26.837469 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gxzdh" event={"ID":"0af98f75-498c-4059-924a-daa3ab0ebfbc","Type":"ContainerStarted","Data":"8571feb6bc3f646516f2be8a9cef78baca7808132aed7111c4975aa5427ba254"} Jul 29 14:35:26.840520 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:26.840416 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" event={"ID":"64bb03c8-45a0-470b-a207-e1e660b6faf2","Type":"ContainerStarted","Data":"3aecb3f40c7b8b0b9cc81f6c04276a49a8a79c0c7c0f7822043311276f36a73c"} Jul 29 14:35:26.845175 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:26.845151 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n84x4" event={"ID":"7c5930f2-ccc2-476a-8d31-9e85d2735b7a","Type":"ContainerStarted","Data":"0463f36df7d8d06314d794d08d0ee94beb535c9a18a6fc08df2c21551ffa317e"} Jul 29 14:35:27.283978 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:27.283942 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs\") pod \"network-metrics-daemon-fmd7q\" (UID: \"1e27a282-f2d3-4aa9-81d3-468c1da0ab8e\") " pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:27.284178 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:27.284122 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 14:35:27.284259 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:27.284182 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs podName:1e27a282-f2d3-4aa9-81d3-468c1da0ab8e nodeName:}" failed. No retries permitted until 2026-07-29 14:35:29.284163887 +0000 UTC m=+6.012313068 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs") pod "network-metrics-daemon-fmd7q" (UID: "1e27a282-f2d3-4aa9-81d3-468c1da0ab8e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 14:35:27.384747 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:27.384299 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zx62\" (UniqueName: \"kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62\") pod \"network-check-target-vv7bl\" (UID: \"10c2f512-d00b-4f28-8957-4311e81257ec\") " pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:27.384747 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:27.384368 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret\") pod \"global-pull-secret-syncer-tfbhf\" (UID: \"3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9\") " pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:27.384747 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:27.384506 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 14:35:27.384747 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:27.384566 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret podName:3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9 nodeName:}" failed. No retries permitted until 2026-07-29 14:35:29.384547284 +0000 UTC m=+6.112696473 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret") pod "global-pull-secret-syncer-tfbhf" (UID: "3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9") : object "kube-system"/"original-pull-secret" not registered Jul 29 14:35:27.385072 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:27.385005 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 14:35:27.385072 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:27.385028 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 14:35:27.385072 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:27.385045 2580 projected.go:196] Error preparing data for projected volume kube-api-access-7zx62 for pod openshift-network-diagnostics/network-check-target-vv7bl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 14:35:27.385261 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:27.385093 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62 podName:10c2f512-d00b-4f28-8957-4311e81257ec nodeName:}" failed. No retries permitted until 2026-07-29 14:35:29.385075896 +0000 UTC m=+6.113225063 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7zx62" (UniqueName: "kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62") pod "network-check-target-vv7bl" (UID: "10c2f512-d00b-4f28-8957-4311e81257ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 14:35:27.705294 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:27.705182 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 14:30:25 +0000 UTC" deadline="2028-02-29 17:51:54.685399603 +0000 UTC" Jul 29 14:35:27.705294 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:27.705215 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13923h16m26.980188431s" Jul 29 14:35:27.812866 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:27.812826 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:27.813024 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:27.812949 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:35:27.813369 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:27.813351 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:27.813479 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:27.813459 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:35:27.816184 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:27.815472 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:27.816184 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:27.815556 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:35:29.298863 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:29.298239 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs\") pod \"network-metrics-daemon-fmd7q\" (UID: \"1e27a282-f2d3-4aa9-81d3-468c1da0ab8e\") " pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:29.298863 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:29.298439 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 14:35:29.298863 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:29.298500 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs podName:1e27a282-f2d3-4aa9-81d3-468c1da0ab8e nodeName:}" failed. No retries permitted until 2026-07-29 14:35:33.298481772 +0000 UTC m=+10.026630959 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs") pod "network-metrics-daemon-fmd7q" (UID: "1e27a282-f2d3-4aa9-81d3-468c1da0ab8e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 14:35:29.399472 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:29.399439 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret\") pod \"global-pull-secret-syncer-tfbhf\" (UID: \"3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9\") " pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:29.399617 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:29.399521 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zx62\" (UniqueName: \"kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62\") pod \"network-check-target-vv7bl\" (UID: \"10c2f512-d00b-4f28-8957-4311e81257ec\") " pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:29.399778 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:29.399686 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 14:35:29.399778 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:29.399733 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 14:35:29.399778 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:29.399747 2580 projected.go:196] Error preparing data for projected volume kube-api-access-7zx62 for pod openshift-network-diagnostics/network-check-target-vv7bl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 14:35:29.399980 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:29.399832 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62 podName:10c2f512-d00b-4f28-8957-4311e81257ec nodeName:}" failed. No retries permitted until 2026-07-29 14:35:33.399795511 +0000 UTC m=+10.127944682 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7zx62" (UniqueName: "kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62") pod "network-check-target-vv7bl" (UID: "10c2f512-d00b-4f28-8957-4311e81257ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 14:35:29.400039 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:29.399996 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 14:35:29.400086 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:29.400039 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret podName:3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9 nodeName:}" failed. No retries permitted until 2026-07-29 14:35:33.400027494 +0000 UTC m=+10.128176674 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret") pod "global-pull-secret-syncer-tfbhf" (UID: "3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9") : object "kube-system"/"original-pull-secret" not registered Jul 29 14:35:29.811890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:29.811859 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:29.812061 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:29.811860 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:29.812061 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:29.811861 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:29.812061 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:29.812008 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:35:29.812212 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:29.812143 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:35:29.812261 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:29.812218 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:35:31.811747 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:31.811605 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:31.811747 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:31.811745 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:31.812237 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:31.811838 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:35:31.812308 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:31.812291 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:31.812427 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:31.812389 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:35:31.812427 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:31.812403 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:35:33.331858 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:33.331321 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs\") pod \"network-metrics-daemon-fmd7q\" (UID: \"1e27a282-f2d3-4aa9-81d3-468c1da0ab8e\") " pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:33.331858 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:33.331475 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 14:35:33.331858 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:33.331534 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs podName:1e27a282-f2d3-4aa9-81d3-468c1da0ab8e nodeName:}" failed. No retries permitted until 2026-07-29 14:35:41.331515722 +0000 UTC m=+18.059664895 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs") pod "network-metrics-daemon-fmd7q" (UID: "1e27a282-f2d3-4aa9-81d3-468c1da0ab8e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 14:35:33.431984 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:33.431831 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret\") pod \"global-pull-secret-syncer-tfbhf\" (UID: \"3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9\") " pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:33.431984 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:33.431925 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zx62\" (UniqueName: \"kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62\") pod \"network-check-target-vv7bl\" (UID: \"10c2f512-d00b-4f28-8957-4311e81257ec\") " pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:33.432207 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:33.432004 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 14:35:33.432207 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:33.432070 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 14:35:33.432207 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:33.432090 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 14:35:33.432207 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:33.432103 2580 projected.go:196] Error preparing data for projected volume kube-api-access-7zx62 for pod openshift-network-diagnostics/network-check-target-vv7bl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 14:35:33.432207 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:33.432076 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret podName:3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9 nodeName:}" failed. No retries permitted until 2026-07-29 14:35:41.432054408 +0000 UTC m=+18.160203588 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret") pod "global-pull-secret-syncer-tfbhf" (UID: "3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9") : object "kube-system"/"original-pull-secret" not registered Jul 29 14:35:33.432207 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:33.432165 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62 podName:10c2f512-d00b-4f28-8957-4311e81257ec nodeName:}" failed. No retries permitted until 2026-07-29 14:35:41.432149708 +0000 UTC m=+18.160298882 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7zx62" (UniqueName: "kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62") pod "network-check-target-vv7bl" (UID: "10c2f512-d00b-4f28-8957-4311e81257ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 14:35:33.813378 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:33.813264 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:33.813560 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:33.813381 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:35:33.813717 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:33.813683 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:33.813872 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:33.813837 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:35:33.813939 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:33.813895 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:33.814002 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:33.813984 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:35:35.811839 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:35.811807 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:35.812259 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:35.811810 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:35.812259 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:35.811939 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:35:35.812259 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:35.811956 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:35.812259 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:35.812063 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:35:35.812259 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:35.812158 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:35:37.811504 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:37.811486 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:37.811850 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:37.811487 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:37.811850 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:37.811606 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:35:37.811850 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:37.811679 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:35:37.811850 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:37.811743 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:37.811850 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:37.811827 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:35:39.812392 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:39.812355 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:39.812392 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:39.812359 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:39.812868 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:39.812517 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:39.812868 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:39.812520 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:35:39.812868 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:39.812623 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:35:39.812868 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:39.812711 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:35:41.399999 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:41.399966 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs\") pod \"network-metrics-daemon-fmd7q\" (UID: \"1e27a282-f2d3-4aa9-81d3-468c1da0ab8e\") " pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:41.400486 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:41.400084 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 14:35:41.400486 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:41.400145 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs podName:1e27a282-f2d3-4aa9-81d3-468c1da0ab8e nodeName:}" failed. No retries permitted until 2026-07-29 14:35:57.400127461 +0000 UTC m=+34.128276645 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs") pod "network-metrics-daemon-fmd7q" (UID: "1e27a282-f2d3-4aa9-81d3-468c1da0ab8e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 14:35:41.500521 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:41.500483 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret\") pod \"global-pull-secret-syncer-tfbhf\" (UID: \"3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9\") " pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:41.500683 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:41.500567 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zx62\" (UniqueName: \"kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62\") pod \"network-check-target-vv7bl\" (UID: \"10c2f512-d00b-4f28-8957-4311e81257ec\") " pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:41.500683 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:41.500609 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 14:35:41.500683 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:41.500685 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret podName:3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9 nodeName:}" failed. No retries permitted until 2026-07-29 14:35:57.500665295 +0000 UTC m=+34.228814486 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret") pod "global-pull-secret-syncer-tfbhf" (UID: "3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9") : object "kube-system"/"original-pull-secret" not registered Jul 29 14:35:41.500873 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:41.500692 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 14:35:41.500873 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:41.500709 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 14:35:41.500873 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:41.500735 2580 projected.go:196] Error preparing data for projected volume kube-api-access-7zx62 for pod openshift-network-diagnostics/network-check-target-vv7bl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 14:35:41.500873 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:41.500787 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62 podName:10c2f512-d00b-4f28-8957-4311e81257ec nodeName:}" failed. No retries permitted until 2026-07-29 14:35:57.500773305 +0000 UTC m=+34.228922486 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7zx62" (UniqueName: "kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62") pod "network-check-target-vv7bl" (UID: "10c2f512-d00b-4f28-8957-4311e81257ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 14:35:41.812178 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:41.812148 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:41.812342 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:41.812263 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:35:41.812342 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:41.812281 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:41.812452 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:41.812346 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:41.812498 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:41.812454 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:35:41.812547 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:41.812531 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:35:43.812235 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:43.812206 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:43.812657 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:43.812294 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:35:43.812657 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:43.812302 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:43.812657 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:43.812329 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:43.812657 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:43.812367 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:35:43.812657 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:43.812466 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:35:45.811998 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.811848 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:45.812559 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.811863 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:45.812559 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:45.812073 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:35:45.812559 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:45.812167 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:35:45.812559 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.811976 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:45.812559 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:45.812267 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:35:45.882348 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.882321 2580 generic.go:356] "Generic (PLEG): container finished" podID="4986a82ee8d2a285c70881170330a296" containerID="cc3d946092b82e0dc50a9e6ef26de890c66ee97d07fbf407bd455c58c293b086" exitCode=0 Jul 29 14:35:45.882460 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.882406 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-42.ec2.internal" event={"ID":"4986a82ee8d2a285c70881170330a296","Type":"ContainerDied","Data":"cc3d946092b82e0dc50a9e6ef26de890c66ee97d07fbf407bd455c58c293b086"} Jul 29 14:35:45.884121 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.884097 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jrs97" event={"ID":"a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d","Type":"ContainerStarted","Data":"a5bdb0a18f2c17a977ec962b74ccd84127b6e17ed12d9a19cccfe22fe06fc774"} Jul 29 14:35:45.885414 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.885395 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" event={"ID":"6b1c56e1-11a4-451e-9982-f4b47335ff9f","Type":"ContainerStarted","Data":"6c12b10bb6359095a85daeffe0be9ed29c260a9bd774d8eb1b457dfa7d7e44e3"} Jul 29 14:35:45.886675 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.886655 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gxzdh" event={"ID":"0af98f75-498c-4059-924a-daa3ab0ebfbc","Type":"ContainerStarted","Data":"e984cab9609e0fa4e86f98e86a67982a7927eb7a9b42fb969440e91140cb24a7"} Jul 29 14:35:45.889398 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.889378 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 14:35:45.889767 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.889749 2580 generic.go:356] "Generic (PLEG): container finished" podID="64bb03c8-45a0-470b-a207-e1e660b6faf2" containerID="eaa8d914d203a20407c404b45aeed9211b01d008787223412e38bdbc9e6d6408" exitCode=1 Jul 29 14:35:45.889829 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.889811 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" event={"ID":"64bb03c8-45a0-470b-a207-e1e660b6faf2","Type":"ContainerStarted","Data":"795efffe7ed74874a43f96d6868c60391db4261743633a66cae0bfd4a19ead18"} Jul 29 14:35:45.889898 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.889883 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" event={"ID":"64bb03c8-45a0-470b-a207-e1e660b6faf2","Type":"ContainerStarted","Data":"82afdb41e47405361fd2dac3a65b1b5a7a2e80c896c3c7fcdef490d8fde93cf0"} Jul 29 14:35:45.889953 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.889936 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" event={"ID":"64bb03c8-45a0-470b-a207-e1e660b6faf2","Type":"ContainerStarted","Data":"4b45f060df5c5aa18d9d71134e10bc5a11ee4e6aa6f57a9c00db6f8fcff65bd4"} Jul 29 14:35:45.889999 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.889970 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" event={"ID":"64bb03c8-45a0-470b-a207-e1e660b6faf2","Type":"ContainerDied","Data":"eaa8d914d203a20407c404b45aeed9211b01d008787223412e38bdbc9e6d6408"} Jul 29 14:35:45.890039 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.890004 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" event={"ID":"64bb03c8-45a0-470b-a207-e1e660b6faf2","Type":"ContainerStarted","Data":"df797547e551eeea3ea3d6d80d2ba0ae25d2a3d95ef3796edd806703b6d73d46"} Jul 29 14:35:45.891173 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.891154 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n84x4" event={"ID":"7c5930f2-ccc2-476a-8d31-9e85d2735b7a","Type":"ContainerStarted","Data":"c22b335795be29b06a364a74cbf95cc9d411ea776a3f1041d2baa95e40b01d1c"} Jul 29 14:35:45.892587 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.892569 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-42.ec2.internal" event={"ID":"ba036abba3605af2a4be113b0020ec88","Type":"ContainerStarted","Data":"1ee15511ec521f66a1736db30531bb924a1a865febae18c3056078cfd93a28b2"} Jul 29 14:35:45.893946 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.893913 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-krcjx" event={"ID":"a1e60e32-5fbd-46a4-9458-fab72b4b4b68","Type":"ContainerStarted","Data":"b018f81c2ab30371b09e68a09cd42caab4b452030dbdb512c3528771d6f86169"} Jul 29 14:35:45.895304 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.895283 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-99w5n" event={"ID":"36c6b7d1-eca5-4c89-a812-ea83e6ba33a7","Type":"ContainerStarted","Data":"1f5b208ab697ea9ff1cc1ef499823c5f93247806f30c9a9ae9edc2b1cc844366"} Jul 29 14:35:45.897044 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.897027 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9dcks" event={"ID":"5cdab631-5539-4ea1-845f-70a030e2c6b2","Type":"ContainerStarted","Data":"8db75409c463f3f9450d1a92b9f6d793dd0d6c9fd2e5253c88ad79223097dcaa"} Jul 29 14:35:45.968483 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.968466 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-krcjx_a1e60e32-5fbd-46a4-9458-fab72b4b4b68/dns-node-resolver/0.log" Jul 29 14:35:45.985523 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.985485 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9dcks" podStartSLOduration=3.70347809 podStartE2EDuration="22.98547199s" totalImagesPullingTime="19.2819939s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:35:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:35:45.965410341 +0000 UTC m=+22.693559530" watchObservedRunningTime="2026-07-29 14:35:45.98547199 +0000 UTC m=+22.713621177" Jul 29 14:35:45.986101 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:45.986072 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-99w5n" podStartSLOduration=11.206284405 podStartE2EDuration="22.986063109s" totalImagesPullingTime="11.779778704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:35:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:35:45.985419696 +0000 UTC m=+22.713568884" watchObservedRunningTime="2026-07-29 14:35:45.986063109 +0000 UTC m=+22.714212297" Jul 29 14:35:46.000651 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:46.000629 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:46.017808 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:46.017707 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-krcjx" podStartSLOduration=3.7713820179999997 podStartE2EDuration="23.017690085s" totalImagesPullingTime="19.246308067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:35:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:35:46.017487607 +0000 UTC m=+22.745636795" watchObservedRunningTime="2026-07-29 14:35:46.017690085 +0000 UTC m=+22.745839274" Jul 29 14:35:46.038920 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:46.038881 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gxzdh" podStartSLOduration=3.800535234 podStartE2EDuration="23.038868123s" totalImagesPullingTime="19.238332889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:35:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:35:46.038640013 +0000 UTC m=+22.766789201" watchObservedRunningTime="2026-07-29 14:35:46.038868123 +0000 UTC m=+22.767017326" Jul 29 14:35:46.059686 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:46.059649 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jrs97" podStartSLOduration=3.776667967 podStartE2EDuration="23.059639065s" totalImagesPullingTime="19.282971098s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:35:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:35:46.058788595 +0000 UTC m=+22.786937786" watchObservedRunningTime="2026-07-29 14:35:46.059639065 +0000 UTC m=+22.787788254" Jul 29 14:35:46.079683 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:46.079637 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-42.ec2.internal" podStartSLOduration=22.079627048 podStartE2EDuration="22.079627048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:35:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:35:46.079364862 +0000 UTC m=+22.807514050" watchObservedRunningTime="2026-07-29 14:35:46.079627048 +0000 UTC m=+22.807776236" Jul 29 14:35:46.111148 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:46.111129 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:46.200753 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:46.200718 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:46.292780 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:46.292698 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:46.598163 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:46.598134 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:46.699092 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:46.699049 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:46.901823 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:46.901759 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 14:35:46.902329 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:46.902169 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" event={"ID":"64bb03c8-45a0-470b-a207-e1e660b6faf2","Type":"ContainerStarted","Data":"e0ee19396f7d4c4684db04605357c123d9fba7f1be9302aab3308c9a05035c04"} Jul 29 14:35:46.903759 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:46.903711 2580 generic.go:356] "Generic (PLEG): container finished" podID="7c5930f2-ccc2-476a-8d31-9e85d2735b7a" containerID="c22b335795be29b06a364a74cbf95cc9d411ea776a3f1041d2baa95e40b01d1c" exitCode=0 Jul 29 14:35:46.903873 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:46.903756 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n84x4" event={"ID":"7c5930f2-ccc2-476a-8d31-9e85d2735b7a","Type":"ContainerDied","Data":"c22b335795be29b06a364a74cbf95cc9d411ea776a3f1041d2baa95e40b01d1c"} Jul 29 14:35:46.905259 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:46.905232 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qmdq4" event={"ID":"5d695c7f-6233-4e98-a4f1-1a35f31ac624","Type":"ContainerStarted","Data":"c4ed8c101a57a8589ccd4fab962155a78f7f1cec67fdc0535ce2a0055bc66f36"} Jul 29 14:35:46.907001 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:46.906933 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-42.ec2.internal" event={"ID":"4986a82ee8d2a285c70881170330a296","Type":"ContainerStarted","Data":"64360d36c55d811f2b57a898dad831cbc78731b913bc27b485a7791ad5946544"} Jul 29 14:35:46.920545 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:46.920498 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qmdq4" podStartSLOduration=4.660313852 podStartE2EDuration="23.920482494s" totalImagesPullingTime="19.260168642s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:35:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:35:46.91986988 +0000 UTC m=+23.648019068" watchObservedRunningTime="2026-07-29 14:35:46.920482494 +0000 UTC m=+23.648631686" Jul 29 14:35:46.933308 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:46.933268 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-42.ec2.internal" podStartSLOduration=22.933255538 podStartE2EDuration="22.933255538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:35:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:35:46.933245152 +0000 UTC m=+23.661394339" watchObservedRunningTime="2026-07-29 14:35:46.933255538 +0000 UTC m=+23.661404727" Jul 29 14:35:47.047386 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:47.047357 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:47.123893 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:47.123865 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gxzdh_0af98f75-498c-4059-924a-daa3ab0ebfbc/node-ca/0.log" Jul 29 14:35:47.145599 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:47.145571 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:47.342669 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:47.342648 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 29 14:35:47.721245 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:47.721109 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T14:35:47.342666645Z","UUID":"c3dc6535-5e98-40c5-afaa-93b1dc7942d8","Handler":null,"Name":"","Endpoint":""} Jul 29 14:35:47.722711 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:47.722689 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 29 14:35:47.722844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:47.722748 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 29 14:35:47.811784 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:47.811757 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:47.811912 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:47.811757 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:47.811912 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:47.811867 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:35:47.812024 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:47.811972 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:35:47.812024 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:47.811764 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:47.812117 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:47.812073 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:35:47.910004 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:47.909974 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" event={"ID":"6b1c56e1-11a4-451e-9982-f4b47335ff9f","Type":"ContainerStarted","Data":"8068a561334e6d3463ea152b617b55d96c6d64c43559e295fb15745a188e5ede"} Jul 29 14:35:47.952488 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:47.952460 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:48.054290 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:48.054263 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:48.915786 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:48.915757 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 14:35:48.916269 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:48.916157 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" event={"ID":"64bb03c8-45a0-470b-a207-e1e660b6faf2","Type":"ContainerStarted","Data":"6c413a83250c338107fbe0130d72fc5a07242eae6f099a7b3effdafe65d45a28"} Jul 29 14:35:49.099071 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:49.099037 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:49.476307 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:49.476277 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:49.576556 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:49.576523 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:49.801845 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:49.801800 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:49.811672 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:49.811640 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:49.811672 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:49.811668 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:49.811862 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:49.811641 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:49.811862 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:49.811774 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:35:49.811862 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:49.811841 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:35:49.812022 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:49.811914 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:35:49.931785 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:49.931754 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:50.361624 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:50.361452 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:50.452065 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:50.452025 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:50.749058 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:50.749026 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:50.878090 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:50.878061 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-99w5n" Jul 29 14:35:50.878600 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:50.878576 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-99w5n" Jul 29 14:35:50.922660 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:50.922628 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" event={"ID":"6b1c56e1-11a4-451e-9982-f4b47335ff9f","Type":"ContainerStarted","Data":"17c0170f1b2b19b7e713e09569a699ae28e624619f4b66a7dd8e29eb592500f9"} Jul 29 14:35:50.926375 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:50.926356 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 14:35:50.926811 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:50.926780 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" event={"ID":"64bb03c8-45a0-470b-a207-e1e660b6faf2","Type":"ContainerStarted","Data":"658d0476be577b337c4c74420a3dc9c69b7d586c077fb66cd9cb4161275d40de"} Jul 29 14:35:50.927072 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:50.927041 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:50.927228 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:50.927211 2580 scope.go:117] "RemoveContainer" containerID="eaa8d914d203a20407c404b45aeed9211b01d008787223412e38bdbc9e6d6408" Jul 29 14:35:50.928881 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:50.928859 2580 generic.go:356] "Generic (PLEG): container finished" podID="7c5930f2-ccc2-476a-8d31-9e85d2735b7a" containerID="1229f57bbae8b01600a11000b319d2328fc3f2067be49c99c0491a9e9303fef8" exitCode=0 Jul 29 14:35:50.928987 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:50.928942 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n84x4" event={"ID":"7c5930f2-ccc2-476a-8d31-9e85d2735b7a","Type":"ContainerDied","Data":"1229f57bbae8b01600a11000b319d2328fc3f2067be49c99c0491a9e9303fef8"} Jul 29 14:35:50.929046 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:50.928999 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-99w5n" Jul 29 14:35:50.929523 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:50.929501 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-99w5n" Jul 29 14:35:50.940018 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:50.939965 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ssdb4" podStartSLOduration=4.565608816 podStartE2EDuration="27.939945109s" totalImagesPullingTime="23.374336293s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:35:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:35:50.939919693 +0000 UTC m=+27.668068882" watchObservedRunningTime="2026-07-29 14:35:50.939945109 +0000 UTC m=+27.668094299" Jul 29 14:35:50.943322 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:50.943302 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:51.360827 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:51.360739 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:51.747153 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:51.747111 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:51.812194 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:51.812175 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:51.812307 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:51.812209 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:51.812307 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:51.812258 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:35:51.812393 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:51.812325 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:35:51.812393 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:51.812372 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:51.812473 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:51.812452 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:35:51.934780 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:51.934755 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 14:35:51.935244 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:51.935217 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" event={"ID":"64bb03c8-45a0-470b-a207-e1e660b6faf2","Type":"ContainerStarted","Data":"bb078d0be0ac5cf508213aec06a60b75e0ccc0c625cae25e5d7762a7c371a0a8"} Jul 29 14:35:51.935512 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:51.935495 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:51.935917 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:51.935889 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:51.947627 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:51.947600 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:51.951157 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:51.951132 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:35:51.964436 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:51.964397 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" podStartSLOduration=9.649456487 podStartE2EDuration="28.964386124s" totalImagesPullingTime="19.314929637s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:35:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:35:51.964047399 +0000 UTC m=+28.692196623" watchObservedRunningTime="2026-07-29 14:35:51.964386124 +0000 UTC m=+28.692535313" Jul 29 14:35:52.348633 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:52.348607 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:52.939433 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:52.939376 2580 generic.go:356] "Generic (PLEG): container finished" podID="7c5930f2-ccc2-476a-8d31-9e85d2735b7a" containerID="20a1ee0bee2cbacea918bc13561803157787f57cc855ade08bb72e33806fa5b7" exitCode=0 Jul 29 14:35:52.939595 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:52.939457 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n84x4" event={"ID":"7c5930f2-ccc2-476a-8d31-9e85d2735b7a","Type":"ContainerDied","Data":"20a1ee0bee2cbacea918bc13561803157787f57cc855ade08bb72e33806fa5b7"} Jul 29 14:35:52.949233 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:52.949200 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:53.134419 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:53.134390 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:53.204743 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:53.204658 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:53.390647 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:53.390610 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:53.529786 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:53.529764 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:53.750925 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:53.750900 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:53.812638 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:53.812592 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:53.812758 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:53.812683 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:35:53.812813 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:53.812797 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:53.812918 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:53.812898 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:35:53.812952 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:53.812933 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:53.813034 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:53.813019 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:35:53.953035 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:53.953008 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:54.053388 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:54.053368 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:54.360570 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:54.360496 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:54.947147 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:54.947114 2580 generic.go:356] "Generic (PLEG): container finished" podID="7c5930f2-ccc2-476a-8d31-9e85d2735b7a" containerID="3ec884750f0fe0777de75039e49a869452334a2f3b9f3673930ad547a7958168" exitCode=0 Jul 29 14:35:54.947259 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:54.947166 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n84x4" event={"ID":"7c5930f2-ccc2-476a-8d31-9e85d2735b7a","Type":"ContainerDied","Data":"3ec884750f0fe0777de75039e49a869452334a2f3b9f3673930ad547a7958168"} Jul 29 14:35:55.151685 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:55.151646 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:55.749175 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:55.748965 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:55.812005 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:55.811977 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:55.812151 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:55.811977 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:55.812151 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:55.812081 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:35:55.812151 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:55.812098 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:55.812305 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:55.812204 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:35:55.812305 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:55.812291 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:35:55.850578 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:55.850554 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:55.949803 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:55.949772 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:56.144884 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:56.144764 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:57.351845 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:57.351810 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:57.422655 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:57.422631 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs\") pod \"network-metrics-daemon-fmd7q\" (UID: \"1e27a282-f2d3-4aa9-81d3-468c1da0ab8e\") " pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:57.422836 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:57.422802 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 14:35:57.422890 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:57.422869 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs podName:1e27a282-f2d3-4aa9-81d3-468c1da0ab8e nodeName:}" failed. No retries permitted until 2026-07-29 14:36:29.422848347 +0000 UTC m=+66.150997528 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs") pod "network-metrics-daemon-fmd7q" (UID: "1e27a282-f2d3-4aa9-81d3-468c1da0ab8e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 14:35:57.523607 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:57.523581 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zx62\" (UniqueName: \"kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62\") pod \"network-check-target-vv7bl\" (UID: \"10c2f512-d00b-4f28-8957-4311e81257ec\") " pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:57.523745 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:57.523616 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret\") pod \"global-pull-secret-syncer-tfbhf\" (UID: \"3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9\") " pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:57.523745 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:57.523707 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 14:35:57.523828 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:57.523742 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 14:35:57.523828 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:57.523759 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 14:35:57.523828 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:57.523762 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret podName:3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9 nodeName:}" failed. No retries permitted until 2026-07-29 14:36:29.523750797 +0000 UTC m=+66.251899965 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret") pod "global-pull-secret-syncer-tfbhf" (UID: "3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9") : object "kube-system"/"original-pull-secret" not registered Jul 29 14:35:57.523828 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:57.523769 2580 projected.go:196] Error preparing data for projected volume kube-api-access-7zx62 for pod openshift-network-diagnostics/network-check-target-vv7bl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 14:35:57.523828 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:57.523805 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62 podName:10c2f512-d00b-4f28-8957-4311e81257ec nodeName:}" failed. No retries permitted until 2026-07-29 14:36:29.523794398 +0000 UTC m=+66.251943581 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7zx62" (UniqueName: "kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62") pod "network-check-target-vv7bl" (UID: "10c2f512-d00b-4f28-8957-4311e81257ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 14:35:57.551325 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:57.551300 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:57.758982 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:57.758950 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:57.811629 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:57.811603 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:57.811783 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:57.811630 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:57.811783 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:57.811750 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:35:57.811888 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:57.811791 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:35:57.811888 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:57.811854 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:57.811986 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:57.811946 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:35:57.952056 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:57.952020 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:58.149823 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:58.149751 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:59.155028 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:59.154996 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:59.555160 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:59.555128 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:35:59.812611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:59.812584 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:35:59.812611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:59.812601 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:35:59.812803 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:59.812711 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:35:59.812803 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:59.812767 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:35:59.812873 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:59.812849 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:35:59.812942 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:35:59.812925 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:35:59.959962 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:59.959930 2580 generic.go:356] "Generic (PLEG): container finished" podID="7c5930f2-ccc2-476a-8d31-9e85d2735b7a" containerID="eb80434947b867f0d7b79693fca8b7445be9b65231fbfc78c5bfd6fc80107577" exitCode=0 Jul 29 14:35:59.960076 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:35:59.959977 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n84x4" event={"ID":"7c5930f2-ccc2-476a-8d31-9e85d2735b7a","Type":"ContainerDied","Data":"eb80434947b867f0d7b79693fca8b7445be9b65231fbfc78c5bfd6fc80107577"} Jul 29 14:36:00.008987 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:00.008958 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:00.126746 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:00.126680 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:00.184655 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:00.184636 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:00.965189 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:00.965163 2580 generic.go:356] "Generic (PLEG): container finished" podID="7c5930f2-ccc2-476a-8d31-9e85d2735b7a" containerID="11c5e9d775d653f24cdc1d5bf935539959b0e7dc3964cde95c690819d86623ca" exitCode=0 Jul 29 14:36:00.965314 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:00.965203 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n84x4" event={"ID":"7c5930f2-ccc2-476a-8d31-9e85d2735b7a","Type":"ContainerDied","Data":"11c5e9d775d653f24cdc1d5bf935539959b0e7dc3964cde95c690819d86623ca"} Jul 29 14:36:01.099956 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:01.099930 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:01.811989 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:01.811964 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:36:01.812384 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:01.811987 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:36:01.812384 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:01.811987 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:36:01.812384 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:36:01.812052 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:36:01.812384 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:36:01.812153 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:36:01.812384 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:36:01.812213 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:36:01.875658 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:01.875630 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:01.970527 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:01.970506 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n84x4" event={"ID":"7c5930f2-ccc2-476a-8d31-9e85d2735b7a","Type":"ContainerStarted","Data":"c053d16a54a2b1f7c0042f590bb1cb38efcda21f762268686a343f848a3a5832"} Jul 29 14:36:01.976242 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:01.976219 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:01.991408 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:01.991366 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-n84x4" podStartSLOduration=8.353911973 podStartE2EDuration="38.991353559s" totalImagesPullingTime="30.637441586s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:35:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:36:01.99087871 +0000 UTC m=+38.719027898" watchObservedRunningTime="2026-07-29 14:36:01.991353559 +0000 UTC m=+38.719502742" Jul 29 14:36:03.812784 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:03.812749 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:36:03.813264 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:03.812821 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:36:03.813264 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:36:03.812832 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:36:03.813264 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:36:03.812952 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:36:03.813264 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:03.812981 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:36:03.813264 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:36:03.813045 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:36:04.195870 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:04.195794 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:05.812513 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:05.812481 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:36:05.812872 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:05.812491 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:36:05.812872 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:36:05.812594 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:36:05.812872 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:36:05.812711 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:36:05.812872 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:05.812796 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:36:05.813009 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:36:05.812894 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:36:07.812372 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:07.812343 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:36:07.812759 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:07.812352 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:36:07.812759 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:36:07.812444 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:36:07.812759 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:36:07.812505 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:36:07.812759 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:07.812549 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:36:07.812759 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:36:07.812600 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:36:08.499593 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:08.499548 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:08.663260 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:08.663229 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:08.762205 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:08.762139 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:08.887518 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:08.887484 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:09.342773 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:09.342741 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vv7bl"] Jul 29 14:36:09.343000 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:09.342852 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:36:09.343000 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:36:09.342940 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:36:09.346173 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:09.346141 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tfbhf"] Jul 29 14:36:09.346308 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:09.346294 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:36:09.346437 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:36:09.346414 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:36:09.349877 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:09.349822 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fmd7q"] Jul 29 14:36:09.350188 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:09.350169 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:36:09.350393 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:36:09.350365 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:36:10.812031 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:10.811836 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:36:10.812364 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:36:10.812100 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:36:11.811860 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:11.811830 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:36:11.811988 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:11.811840 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:36:11.811988 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:36:11.811936 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfbhf" podUID="3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9" Jul 29 14:36:11.812064 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:36:11.812031 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fmd7q" podUID="1e27a282-f2d3-4aa9-81d3-468c1da0ab8e" Jul 29 14:36:12.812090 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:12.812063 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:36:12.812448 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:36:12.812158 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vv7bl" podUID="10c2f512-d00b-4f28-8957-4311e81257ec" Jul 29 14:36:13.063924 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.063870 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-42.ec2.internal" event="NodeReady" Jul 29 14:36:13.064035 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.063961 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 29 14:36:13.111385 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.111358 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-847ddf4c6-zlbq6"] Jul 29 14:36:13.136748 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.136709 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-t9mkp"] Jul 29 14:36:13.136859 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.136840 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.139304 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.139284 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 29 14:36:13.139500 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.139486 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 29 14:36:13.139577 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.139518 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 29 14:36:13.140841 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.140791 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hfmq9\"" Jul 29 14:36:13.147967 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.147940 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 29 14:36:13.150972 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.150939 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-847ddf4c6-zlbq6"] Jul 29 14:36:13.151056 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.150979 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hm2lz"] Jul 29 14:36:13.151100 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.151051 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t9mkp" Jul 29 14:36:13.153910 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.153890 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 29 14:36:13.154130 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.154112 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 29 14:36:13.154242 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.154209 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 29 14:36:13.154323 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.154307 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zhf56\"" Jul 29 14:36:13.154452 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.154419 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 29 14:36:13.172927 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.172907 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t9mkp"] Jul 29 14:36:13.173008 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.172939 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hm2lz"] Jul 29 14:36:13.173047 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.173011 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hm2lz" Jul 29 14:36:13.174924 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.174908 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 29 14:36:13.174924 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.174920 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 29 14:36:13.175055 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.174959 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pp4lb\"" Jul 29 14:36:13.232395 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.232371 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4wglj"] Jul 29 14:36:13.238880 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.238862 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/814fcac9-b1b5-4d9f-9146-d06195946b47-trusted-ca\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.238953 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.238887 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57cd5914-9960-4c9f-a4f0-02069a80d502-tmp-dir\") pod \"dns-default-hm2lz\" (UID: \"57cd5914-9960-4c9f-a4f0-02069a80d502\") " pod="openshift-dns/dns-default-hm2lz" Jul 29 14:36:13.238953 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.238903 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/814fcac9-b1b5-4d9f-9146-d06195946b47-image-registry-private-configuration\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.238953 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.238919 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/814fcac9-b1b5-4d9f-9146-d06195946b47-ca-trust-extracted\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.239065 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.239004 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/407b965c-f652-4b4b-b8be-0940e599da7b-crio-socket\") pod \"insights-runtime-extractor-t9mkp\" (UID: \"407b965c-f652-4b4b-b8be-0940e599da7b\") " pod="openshift-insights/insights-runtime-extractor-t9mkp" Jul 29 14:36:13.239065 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.239035 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/814fcac9-b1b5-4d9f-9146-d06195946b47-registry-certificates\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.239133 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.239064 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/814fcac9-b1b5-4d9f-9146-d06195946b47-bound-sa-token\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.239133 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.239114 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/814fcac9-b1b5-4d9f-9146-d06195946b47-installation-pull-secrets\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.239211 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.239132 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57cd5914-9960-4c9f-a4f0-02069a80d502-metrics-tls\") pod \"dns-default-hm2lz\" (UID: \"57cd5914-9960-4c9f-a4f0-02069a80d502\") " pod="openshift-dns/dns-default-hm2lz" Jul 29 14:36:13.239211 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.239148 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45mlj\" (UniqueName: \"kubernetes.io/projected/814fcac9-b1b5-4d9f-9146-d06195946b47-kube-api-access-45mlj\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.239211 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.239167 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/407b965c-f652-4b4b-b8be-0940e599da7b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t9mkp\" (UID: \"407b965c-f652-4b4b-b8be-0940e599da7b\") " pod="openshift-insights/insights-runtime-extractor-t9mkp" Jul 29 14:36:13.239211 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.239180 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg5hh\" (UniqueName: \"kubernetes.io/projected/407b965c-f652-4b4b-b8be-0940e599da7b-kube-api-access-rg5hh\") pod \"insights-runtime-extractor-t9mkp\" (UID: \"407b965c-f652-4b4b-b8be-0940e599da7b\") " pod="openshift-insights/insights-runtime-extractor-t9mkp" Jul 29 14:36:13.239211 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.239195 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/814fcac9-b1b5-4d9f-9146-d06195946b47-registry-tls\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.239370 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.239215 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/407b965c-f652-4b4b-b8be-0940e599da7b-data-volume\") pod \"insights-runtime-extractor-t9mkp\" (UID: \"407b965c-f652-4b4b-b8be-0940e599da7b\") " pod="openshift-insights/insights-runtime-extractor-t9mkp" Jul 29 14:36:13.239370 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.239237 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/407b965c-f652-4b4b-b8be-0940e599da7b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t9mkp\" (UID: \"407b965c-f652-4b4b-b8be-0940e599da7b\") " pod="openshift-insights/insights-runtime-extractor-t9mkp" Jul 29 14:36:13.239370 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.239260 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlfg5\" (UniqueName: \"kubernetes.io/projected/57cd5914-9960-4c9f-a4f0-02069a80d502-kube-api-access-tlfg5\") pod \"dns-default-hm2lz\" (UID: \"57cd5914-9960-4c9f-a4f0-02069a80d502\") " pod="openshift-dns/dns-default-hm2lz" Jul 29 14:36:13.239370 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.239294 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/57cd5914-9960-4c9f-a4f0-02069a80d502-config-volume\") pod \"dns-default-hm2lz\" (UID: \"57cd5914-9960-4c9f-a4f0-02069a80d502\") " pod="openshift-dns/dns-default-hm2lz" Jul 29 14:36:13.240790 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.240779 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4wglj" Jul 29 14:36:13.242665 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.242649 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fb2k5\"" Jul 29 14:36:13.242747 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.242677 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 29 14:36:13.242747 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.242689 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 29 14:36:13.242898 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.242884 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 29 14:36:13.244588 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.244567 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4wglj"] Jul 29 14:36:13.246092 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.246068 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:13.308545 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.308523 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:13.340133 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340090 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/814fcac9-b1b5-4d9f-9146-d06195946b47-trusted-ca\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.340133 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340114 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e841cbf-1150-4f3c-9c60-97f0c2efa9e2-cert\") pod \"ingress-canary-4wglj\" (UID: \"7e841cbf-1150-4f3c-9c60-97f0c2efa9e2\") " pod="openshift-ingress-canary/ingress-canary-4wglj" Jul 29 14:36:13.340250 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340133 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57cd5914-9960-4c9f-a4f0-02069a80d502-tmp-dir\") pod \"dns-default-hm2lz\" (UID: \"57cd5914-9960-4c9f-a4f0-02069a80d502\") " pod="openshift-dns/dns-default-hm2lz" Jul 29 14:36:13.340250 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340151 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbj59\" (UniqueName: \"kubernetes.io/projected/7e841cbf-1150-4f3c-9c60-97f0c2efa9e2-kube-api-access-vbj59\") pod \"ingress-canary-4wglj\" (UID: \"7e841cbf-1150-4f3c-9c60-97f0c2efa9e2\") " pod="openshift-ingress-canary/ingress-canary-4wglj" Jul 29 14:36:13.340250 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340174 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/814fcac9-b1b5-4d9f-9146-d06195946b47-image-registry-private-configuration\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.340250 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340198 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/814fcac9-b1b5-4d9f-9146-d06195946b47-ca-trust-extracted\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.340250 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340236 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/407b965c-f652-4b4b-b8be-0940e599da7b-crio-socket\") pod \"insights-runtime-extractor-t9mkp\" (UID: \"407b965c-f652-4b4b-b8be-0940e599da7b\") " pod="openshift-insights/insights-runtime-extractor-t9mkp" Jul 29 14:36:13.340442 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340306 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/407b965c-f652-4b4b-b8be-0940e599da7b-crio-socket\") pod \"insights-runtime-extractor-t9mkp\" (UID: \"407b965c-f652-4b4b-b8be-0940e599da7b\") " pod="openshift-insights/insights-runtime-extractor-t9mkp" Jul 29 14:36:13.340442 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340347 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/814fcac9-b1b5-4d9f-9146-d06195946b47-registry-certificates\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.340442 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340384 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/814fcac9-b1b5-4d9f-9146-d06195946b47-bound-sa-token\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.340442 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340433 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/814fcac9-b1b5-4d9f-9146-d06195946b47-installation-pull-secrets\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.340629 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340460 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57cd5914-9960-4c9f-a4f0-02069a80d502-metrics-tls\") pod \"dns-default-hm2lz\" (UID: \"57cd5914-9960-4c9f-a4f0-02069a80d502\") " pod="openshift-dns/dns-default-hm2lz" Jul 29 14:36:13.340629 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340487 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45mlj\" (UniqueName: \"kubernetes.io/projected/814fcac9-b1b5-4d9f-9146-d06195946b47-kube-api-access-45mlj\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.340629 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340522 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/57cd5914-9960-4c9f-a4f0-02069a80d502-tmp-dir\") pod \"dns-default-hm2lz\" (UID: \"57cd5914-9960-4c9f-a4f0-02069a80d502\") " pod="openshift-dns/dns-default-hm2lz" Jul 29 14:36:13.340629 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340526 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/407b965c-f652-4b4b-b8be-0940e599da7b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t9mkp\" (UID: \"407b965c-f652-4b4b-b8be-0940e599da7b\") " pod="openshift-insights/insights-runtime-extractor-t9mkp" Jul 29 14:36:13.340629 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340554 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg5hh\" (UniqueName: \"kubernetes.io/projected/407b965c-f652-4b4b-b8be-0940e599da7b-kube-api-access-rg5hh\") pod \"insights-runtime-extractor-t9mkp\" (UID: \"407b965c-f652-4b4b-b8be-0940e599da7b\") " pod="openshift-insights/insights-runtime-extractor-t9mkp" Jul 29 14:36:13.340629 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340583 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/814fcac9-b1b5-4d9f-9146-d06195946b47-registry-tls\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.340629 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340619 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/407b965c-f652-4b4b-b8be-0940e599da7b-data-volume\") pod \"insights-runtime-extractor-t9mkp\" (UID: \"407b965c-f652-4b4b-b8be-0940e599da7b\") " pod="openshift-insights/insights-runtime-extractor-t9mkp" Jul 29 14:36:13.340978 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340648 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/407b965c-f652-4b4b-b8be-0940e599da7b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t9mkp\" (UID: \"407b965c-f652-4b4b-b8be-0940e599da7b\") " pod="openshift-insights/insights-runtime-extractor-t9mkp" Jul 29 14:36:13.340978 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340666 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/814fcac9-b1b5-4d9f-9146-d06195946b47-ca-trust-extracted\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.340978 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340681 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlfg5\" (UniqueName: \"kubernetes.io/projected/57cd5914-9960-4c9f-a4f0-02069a80d502-kube-api-access-tlfg5\") pod \"dns-default-hm2lz\" (UID: \"57cd5914-9960-4c9f-a4f0-02069a80d502\") " pod="openshift-dns/dns-default-hm2lz" Jul 29 14:36:13.340978 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.340769 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/57cd5914-9960-4c9f-a4f0-02069a80d502-config-volume\") pod \"dns-default-hm2lz\" (UID: \"57cd5914-9960-4c9f-a4f0-02069a80d502\") " pod="openshift-dns/dns-default-hm2lz" Jul 29 14:36:13.341178 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.341049 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/407b965c-f652-4b4b-b8be-0940e599da7b-data-volume\") pod \"insights-runtime-extractor-t9mkp\" (UID: \"407b965c-f652-4b4b-b8be-0940e599da7b\") " pod="openshift-insights/insights-runtime-extractor-t9mkp" Jul 29 14:36:13.341539 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.341232 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/814fcac9-b1b5-4d9f-9146-d06195946b47-registry-certificates\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.341861 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.341800 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/814fcac9-b1b5-4d9f-9146-d06195946b47-trusted-ca\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.341991 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.341970 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/57cd5914-9960-4c9f-a4f0-02069a80d502-config-volume\") pod \"dns-default-hm2lz\" (UID: \"57cd5914-9960-4c9f-a4f0-02069a80d502\") " pod="openshift-dns/dns-default-hm2lz" Jul 29 14:36:13.342442 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.342364 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/407b965c-f652-4b4b-b8be-0940e599da7b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t9mkp\" (UID: \"407b965c-f652-4b4b-b8be-0940e599da7b\") " pod="openshift-insights/insights-runtime-extractor-t9mkp" Jul 29 14:36:13.344825 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.344802 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/814fcac9-b1b5-4d9f-9146-d06195946b47-image-registry-private-configuration\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.344825 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.344811 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/57cd5914-9960-4c9f-a4f0-02069a80d502-metrics-tls\") pod \"dns-default-hm2lz\" (UID: \"57cd5914-9960-4c9f-a4f0-02069a80d502\") " pod="openshift-dns/dns-default-hm2lz" Jul 29 14:36:13.344950 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.344844 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/407b965c-f652-4b4b-b8be-0940e599da7b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t9mkp\" (UID: \"407b965c-f652-4b4b-b8be-0940e599da7b\") " pod="openshift-insights/insights-runtime-extractor-t9mkp" Jul 29 14:36:13.344988 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.344950 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/814fcac9-b1b5-4d9f-9146-d06195946b47-registry-tls\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.345043 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.345024 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/814fcac9-b1b5-4d9f-9146-d06195946b47-installation-pull-secrets\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.348155 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.348131 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/814fcac9-b1b5-4d9f-9146-d06195946b47-bound-sa-token\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.348403 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.348387 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg5hh\" (UniqueName: \"kubernetes.io/projected/407b965c-f652-4b4b-b8be-0940e599da7b-kube-api-access-rg5hh\") pod \"insights-runtime-extractor-t9mkp\" (UID: \"407b965c-f652-4b4b-b8be-0940e599da7b\") " pod="openshift-insights/insights-runtime-extractor-t9mkp" Jul 29 14:36:13.348715 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.348693 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlfg5\" (UniqueName: \"kubernetes.io/projected/57cd5914-9960-4c9f-a4f0-02069a80d502-kube-api-access-tlfg5\") pod \"dns-default-hm2lz\" (UID: \"57cd5914-9960-4c9f-a4f0-02069a80d502\") " pod="openshift-dns/dns-default-hm2lz" Jul 29 14:36:13.348829 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.348815 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45mlj\" (UniqueName: \"kubernetes.io/projected/814fcac9-b1b5-4d9f-9146-d06195946b47-kube-api-access-45mlj\") pod \"image-registry-847ddf4c6-zlbq6\" (UID: \"814fcac9-b1b5-4d9f-9146-d06195946b47\") " pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.441781 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.441757 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e841cbf-1150-4f3c-9c60-97f0c2efa9e2-cert\") pod \"ingress-canary-4wglj\" (UID: \"7e841cbf-1150-4f3c-9c60-97f0c2efa9e2\") " pod="openshift-ingress-canary/ingress-canary-4wglj" Jul 29 14:36:13.441855 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.441785 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbj59\" (UniqueName: \"kubernetes.io/projected/7e841cbf-1150-4f3c-9c60-97f0c2efa9e2-kube-api-access-vbj59\") pod \"ingress-canary-4wglj\" (UID: \"7e841cbf-1150-4f3c-9c60-97f0c2efa9e2\") " pod="openshift-ingress-canary/ingress-canary-4wglj" Jul 29 14:36:13.443735 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.443696 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e841cbf-1150-4f3c-9c60-97f0c2efa9e2-cert\") pod \"ingress-canary-4wglj\" (UID: \"7e841cbf-1150-4f3c-9c60-97f0c2efa9e2\") " pod="openshift-ingress-canary/ingress-canary-4wglj" Jul 29 14:36:13.449764 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.449746 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbj59\" (UniqueName: \"kubernetes.io/projected/7e841cbf-1150-4f3c-9c60-97f0c2efa9e2-kube-api-access-vbj59\") pod \"ingress-canary-4wglj\" (UID: \"7e841cbf-1150-4f3c-9c60-97f0c2efa9e2\") " pod="openshift-ingress-canary/ingress-canary-4wglj" Jul 29 14:36:13.450508 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.450491 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:13.459464 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.459446 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t9mkp" Jul 29 14:36:13.480601 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.480583 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hm2lz" Jul 29 14:36:13.549717 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.549689 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4wglj" Jul 29 14:36:13.678752 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.678045 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t9mkp"] Jul 29 14:36:13.678752 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.678156 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hm2lz"] Jul 29 14:36:13.683212 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.683096 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-847ddf4c6-zlbq6"] Jul 29 14:36:13.686358 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:36:13.686336 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57cd5914_9960_4c9f_a4f0_02069a80d502.slice/crio-b7f0508eba8c8cc2f9119f333aac81b8733990368bbc0deab43c7b11b20066d9 WatchSource:0}: Error finding container b7f0508eba8c8cc2f9119f333aac81b8733990368bbc0deab43c7b11b20066d9: Status 404 returned error can't find the container with id b7f0508eba8c8cc2f9119f333aac81b8733990368bbc0deab43c7b11b20066d9 Jul 29 14:36:13.708611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.708589 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4wglj"] Jul 29 14:36:13.711484 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:36:13.711464 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e841cbf_1150_4f3c_9c60_97f0c2efa9e2.slice/crio-c2721bb68906be97b87c44a33d54335fa84fc33bc28d5634634130ca899fa500 WatchSource:0}: Error finding container c2721bb68906be97b87c44a33d54335fa84fc33bc28d5634634130ca899fa500: Status 404 returned error can't find the container with id c2721bb68906be97b87c44a33d54335fa84fc33bc28d5634634130ca899fa500 Jul 29 14:36:13.813304 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.813277 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:36:13.813831 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.813472 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:36:13.815745 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.815713 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 29 14:36:13.816018 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.816004 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nzccd\"" Jul 29 14:36:13.816071 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.816055 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 29 14:36:13.998363 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.998334 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hm2lz" event={"ID":"57cd5914-9960-4c9f-a4f0-02069a80d502","Type":"ContainerStarted","Data":"b7f0508eba8c8cc2f9119f333aac81b8733990368bbc0deab43c7b11b20066d9"} Jul 29 14:36:13.999503 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:13.999479 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t9mkp" event={"ID":"407b965c-f652-4b4b-b8be-0940e599da7b","Type":"ContainerStarted","Data":"14e2abf219c5ec8c5b2aeb9073641ac773f355befe8f12454e14b9f4233d0888"} Jul 29 14:36:14.000685 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:14.000662 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4wglj" event={"ID":"7e841cbf-1150-4f3c-9c60-97f0c2efa9e2","Type":"ContainerStarted","Data":"c2721bb68906be97b87c44a33d54335fa84fc33bc28d5634634130ca899fa500"} Jul 29 14:36:14.002018 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:14.001994 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" event={"ID":"814fcac9-b1b5-4d9f-9146-d06195946b47","Type":"ContainerStarted","Data":"941bf3d686389a62c9e1fd26bd520ae6e4d30e3150026afa4ddcb4bddb9f0487"} Jul 29 14:36:14.002106 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:14.002031 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" event={"ID":"814fcac9-b1b5-4d9f-9146-d06195946b47","Type":"ContainerStarted","Data":"d87ff1b105c61fce0dd4542cbcf115d281d663f243748e6a8357e965da88dccd"} Jul 29 14:36:14.002170 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:14.002136 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:14.024300 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:14.024237 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" podStartSLOduration=6.024101717 podStartE2EDuration="6.024101717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:36:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:36:14.023812403 +0000 UTC m=+50.751961613" watchObservedRunningTime="2026-07-29 14:36:14.024101717 +0000 UTC m=+50.752250888" Jul 29 14:36:14.483353 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:14.483261 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:14.540305 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:14.540278 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:14.812409 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:14.812373 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:36:14.816840 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:14.816555 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 29 14:36:14.816840 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:14.816804 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ht8tz\"" Jul 29 14:36:14.817996 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:14.817195 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 29 14:36:15.314761 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:15.314715 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:15.419550 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:15.419510 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:15.448980 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:15.448946 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-vwwj2"] Jul 29 14:36:15.461574 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:15.461551 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-vwwj2"] Jul 29 14:36:15.461700 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:15.461661 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-vwwj2" Jul 29 14:36:15.463978 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:15.463909 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 29 14:36:15.464109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:15.464045 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 29 14:36:15.464185 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:15.464167 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-drx46\"" Jul 29 14:36:15.515287 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:15.515259 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:15.559140 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:15.559113 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdgbk\" (UniqueName: \"kubernetes.io/projected/16bddc73-bd54-4f98-9041-5b0848562b4a-kube-api-access-wdgbk\") pod \"downloads-7c97dc4555-vwwj2\" (UID: \"16bddc73-bd54-4f98-9041-5b0848562b4a\") " pod="openshift-console/downloads-7c97dc4555-vwwj2" Jul 29 14:36:15.577769 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:15.577699 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:15.659999 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:15.659972 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdgbk\" (UniqueName: \"kubernetes.io/projected/16bddc73-bd54-4f98-9041-5b0848562b4a-kube-api-access-wdgbk\") pod \"downloads-7c97dc4555-vwwj2\" (UID: \"16bddc73-bd54-4f98-9041-5b0848562b4a\") " pod="openshift-console/downloads-7c97dc4555-vwwj2" Jul 29 14:36:15.670552 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:15.670531 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdgbk\" (UniqueName: \"kubernetes.io/projected/16bddc73-bd54-4f98-9041-5b0848562b4a-kube-api-access-wdgbk\") pod \"downloads-7c97dc4555-vwwj2\" (UID: \"16bddc73-bd54-4f98-9041-5b0848562b4a\") " pod="openshift-console/downloads-7c97dc4555-vwwj2" Jul 29 14:36:15.773131 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:15.773106 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-vwwj2" Jul 29 14:36:16.029228 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:16.029194 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:16.133247 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:16.133220 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:16.616680 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:16.616649 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:16.894467 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:16.894294 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:16.991796 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:16.991760 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:17.171162 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:17.171081 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:17.234971 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:17.234940 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:17.334216 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:17.334183 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:17.904471 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:17.904436 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:19.365973 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:19.365940 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:19.468985 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:19.468898 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:20.214142 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:20.214120 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-vwwj2"] Jul 29 14:36:20.217390 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:36:20.217322 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16bddc73_bd54_4f98_9041_5b0848562b4a.slice/crio-1fde89ffc090e5090397f66167c5978602b7bb90e2eb9be8e382e1841e7b54eb WatchSource:0}: Error finding container 1fde89ffc090e5090397f66167c5978602b7bb90e2eb9be8e382e1841e7b54eb: Status 404 returned error can't find the container with id 1fde89ffc090e5090397f66167c5978602b7bb90e2eb9be8e382e1841e7b54eb Jul 29 14:36:21.027317 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:21.027271 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hm2lz" event={"ID":"57cd5914-9960-4c9f-a4f0-02069a80d502","Type":"ContainerStarted","Data":"ad45b3ea2654adec91586cae9b312921059289d6ec2c668d1921ded2bb19df09"} Jul 29 14:36:21.027820 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:21.027348 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hm2lz" event={"ID":"57cd5914-9960-4c9f-a4f0-02069a80d502","Type":"ContainerStarted","Data":"dba4377f4adbab47e197abf523c5b41ddba9d46bb2d0153039ff1909adc03dde"} Jul 29 14:36:21.027820 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:21.027401 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hm2lz" Jul 29 14:36:21.029519 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:21.029485 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t9mkp" event={"ID":"407b965c-f652-4b4b-b8be-0940e599da7b","Type":"ContainerStarted","Data":"281c3aa9572ddeeef0742ad46f334f28f834a3ae45d613166ff789952ab79b5c"} Jul 29 14:36:21.031011 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:21.030989 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-vwwj2" event={"ID":"16bddc73-bd54-4f98-9041-5b0848562b4a","Type":"ContainerStarted","Data":"1fde89ffc090e5090397f66167c5978602b7bb90e2eb9be8e382e1841e7b54eb"} Jul 29 14:36:21.032514 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:21.032490 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4wglj" event={"ID":"7e841cbf-1150-4f3c-9c60-97f0c2efa9e2","Type":"ContainerStarted","Data":"b05c52e2b25a3763e237fc383dc8be28a257c7f4a74fca9535dbdb29bf5bf2ff"} Jul 29 14:36:21.042803 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:21.042679 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hm2lz" podStartSLOduration=1.642684169 podStartE2EDuration="8.042666392s" totalImagesPullingTime="6.399982223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:36:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:36:21.042461575 +0000 UTC m=+57.770610785" watchObservedRunningTime="2026-07-29 14:36:21.042666392 +0000 UTC m=+57.770815559" Jul 29 14:36:21.057335 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:21.057289 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4wglj" podStartSLOduration=1.681208019 podStartE2EDuration="8.057275837s" totalImagesPullingTime="6.376067818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:36:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:36:21.056560347 +0000 UTC m=+57.784709559" watchObservedRunningTime="2026-07-29 14:36:21.057275837 +0000 UTC m=+57.785425004" Jul 29 14:36:23.139691 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:23.139647 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:23.743145 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:23.743113 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:23.844511 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:23.844458 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:23.954230 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:23.954202 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-67v2n" Jul 29 14:36:24.121410 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:24.121313 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:24.180154 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:24.180124 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:26.053199 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:26.053127 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t9mkp" event={"ID":"407b965c-f652-4b4b-b8be-0940e599da7b","Type":"ContainerStarted","Data":"f9eb75e35bd047dfc39aad486ef20ef41db25ef2a939932a232ad50ba92e47bc"} Jul 29 14:36:27.735339 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:27.735301 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:28.433130 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:28.433082 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:28.533458 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:28.533416 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:28.634176 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:28.634040 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:28.762210 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:28.762127 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:29.462681 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:29.462652 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs\") pod \"network-metrics-daemon-fmd7q\" (UID: \"1e27a282-f2d3-4aa9-81d3-468c1da0ab8e\") " pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:36:29.465096 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:29.465074 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 29 14:36:29.475766 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:29.475698 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e27a282-f2d3-4aa9-81d3-468c1da0ab8e-metrics-certs\") pod \"network-metrics-daemon-fmd7q\" (UID: \"1e27a282-f2d3-4aa9-81d3-468c1da0ab8e\") " pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:36:29.563196 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:29.563164 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zx62\" (UniqueName: \"kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62\") pod \"network-check-target-vv7bl\" (UID: \"10c2f512-d00b-4f28-8957-4311e81257ec\") " pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:36:29.563309 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:29.563229 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret\") pod \"global-pull-secret-syncer-tfbhf\" (UID: \"3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9\") " pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:36:29.565393 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:29.565369 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 29 14:36:29.565509 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:29.565419 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 29 14:36:29.576178 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:29.576156 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9-original-pull-secret\") pod \"global-pull-secret-syncer-tfbhf\" (UID: \"3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9\") " pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:36:29.576635 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:29.576612 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 29 14:36:29.586517 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:29.586496 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zx62\" (UniqueName: \"kubernetes.io/projected/10c2f512-d00b-4f28-8957-4311e81257ec-kube-api-access-7zx62\") pod \"network-check-target-vv7bl\" (UID: \"10c2f512-d00b-4f28-8957-4311e81257ec\") " pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:36:29.722104 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:29.722077 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfbhf" Jul 29 14:36:29.729543 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:29.729492 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nzccd\"" Jul 29 14:36:29.737623 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:29.737604 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fmd7q" Jul 29 14:36:29.829713 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:29.829658 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ht8tz\"" Jul 29 14:36:29.838440 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:29.838419 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:36:29.862178 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:29.862144 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tfbhf"] Jul 29 14:36:29.864981 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:36:29.864942 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d2a7a6e_aa3e_44c4_820e_c46d39b7e6f9.slice/crio-c37dc170172388877f5302161685380c1bbf6272cfdef12409b314fc8e706072 WatchSource:0}: Error finding container c37dc170172388877f5302161685380c1bbf6272cfdef12409b314fc8e706072: Status 404 returned error can't find the container with id c37dc170172388877f5302161685380c1bbf6272cfdef12409b314fc8e706072 Jul 29 14:36:29.877291 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:29.877266 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fmd7q"] Jul 29 14:36:29.880680 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:36:29.880641 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e27a282_f2d3_4aa9_81d3_468c1da0ab8e.slice/crio-b809ecb7879a06ddad8109c69892f36b3377082eb9985ed9a6ebfc9a9329b64a WatchSource:0}: Error finding container b809ecb7879a06ddad8109c69892f36b3377082eb9985ed9a6ebfc9a9329b64a: Status 404 returned error can't find the container with id b809ecb7879a06ddad8109c69892f36b3377082eb9985ed9a6ebfc9a9329b64a Jul 29 14:36:29.957818 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:29.957787 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:29.968032 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:29.968008 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vv7bl"] Jul 29 14:36:29.971946 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:36:29.971916 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10c2f512_d00b_4f28_8957_4311e81257ec.slice/crio-fb2b7552dceb1a506f01f1c9e1ad3072f36e10c1966001827cf554061fa9978c WatchSource:0}: Error finding container fb2b7552dceb1a506f01f1c9e1ad3072f36e10c1966001827cf554061fa9978c: Status 404 returned error can't find the container with id fb2b7552dceb1a506f01f1c9e1ad3072f36e10c1966001827cf554061fa9978c Jul 29 14:36:30.059766 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.059710 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:30.067410 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.067384 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tfbhf" event={"ID":"3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9","Type":"ContainerStarted","Data":"c37dc170172388877f5302161685380c1bbf6272cfdef12409b314fc8e706072"} Jul 29 14:36:30.068652 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.068603 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fmd7q" event={"ID":"1e27a282-f2d3-4aa9-81d3-468c1da0ab8e","Type":"ContainerStarted","Data":"b809ecb7879a06ddad8109c69892f36b3377082eb9985ed9a6ebfc9a9329b64a"} Jul 29 14:36:30.070872 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.070842 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t9mkp" event={"ID":"407b965c-f652-4b4b-b8be-0940e599da7b","Type":"ContainerStarted","Data":"1e94a9014099303e2a2729e0e2803d535058e0bf24084cce2cc171cdb689decc"} Jul 29 14:36:30.072113 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.072093 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vv7bl" event={"ID":"10c2f512-d00b-4f28-8957-4311e81257ec","Type":"ContainerStarted","Data":"fb2b7552dceb1a506f01f1c9e1ad3072f36e10c1966001827cf554061fa9978c"} Jul 29 14:36:30.091962 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.091530 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-t9mkp" podStartSLOduration=1.717627939 podStartE2EDuration="17.091511978s" totalImagesPullingTime="15.373884039s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:36:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:36:30.090582773 +0000 UTC m=+66.818731962" watchObservedRunningTime="2026-07-29 14:36:30.091511978 +0000 UTC m=+66.819661171" Jul 29 14:36:30.491220 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.491100 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:30.575264 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.575213 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:30.681565 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.681496 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:30.701436 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.701408 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd"] Jul 29 14:36:30.703777 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.703757 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd" Jul 29 14:36:30.708364 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.708338 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kqphh"] Jul 29 14:36:30.709665 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.709637 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 29 14:36:30.709810 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.709746 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 29 14:36:30.709915 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.709894 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 29 14:36:30.710017 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.709998 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-pfdg8\"" Jul 29 14:36:30.710087 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.710072 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 29 14:36:30.710395 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.710368 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 29 14:36:30.710828 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.710810 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.716297 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.715620 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 29 14:36:30.716297 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.715860 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5nz5x\"" Jul 29 14:36:30.716297 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.715880 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 29 14:36:30.716297 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.716106 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 29 14:36:30.725556 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.725517 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd"] Jul 29 14:36:30.772006 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.771973 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4162058-cbae-40d7-b6c2-e2a3133fd372-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-27chd\" (UID: \"f4162058-cbae-40d7-b6c2-e2a3133fd372\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd" Jul 29 14:36:30.772151 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.772019 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4396d907-2db8-4122-9e06-0c84c2977b7d-sys\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.772151 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.772051 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4396d907-2db8-4122-9e06-0c84c2977b7d-node-exporter-textfile\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.772151 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.772089 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4396d907-2db8-4122-9e06-0c84c2977b7d-root\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.772151 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.772116 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4396d907-2db8-4122-9e06-0c84c2977b7d-node-exporter-tls\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.772151 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.772144 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4396d907-2db8-4122-9e06-0c84c2977b7d-node-exporter-wtmp\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.772441 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.772169 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4396d907-2db8-4122-9e06-0c84c2977b7d-node-exporter-accelerators-collector-config\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.772441 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.772197 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4396d907-2db8-4122-9e06-0c84c2977b7d-metrics-client-ca\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.772441 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.772231 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4162058-cbae-40d7-b6c2-e2a3133fd372-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-27chd\" (UID: \"f4162058-cbae-40d7-b6c2-e2a3133fd372\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd" Jul 29 14:36:30.772441 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.772273 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4396d907-2db8-4122-9e06-0c84c2977b7d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.772441 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.772300 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6r8x\" (UniqueName: \"kubernetes.io/projected/4396d907-2db8-4122-9e06-0c84c2977b7d-kube-api-access-l6r8x\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.772441 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.772333 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4162058-cbae-40d7-b6c2-e2a3133fd372-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-27chd\" (UID: \"f4162058-cbae-40d7-b6c2-e2a3133fd372\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd" Jul 29 14:36:30.772441 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.772361 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64qg2\" (UniqueName: \"kubernetes.io/projected/f4162058-cbae-40d7-b6c2-e2a3133fd372-kube-api-access-64qg2\") pod \"openshift-state-metrics-b8f49f5c6-27chd\" (UID: \"f4162058-cbae-40d7-b6c2-e2a3133fd372\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd" Jul 29 14:36:30.793958 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.793930 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:30.873746 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.872691 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4396d907-2db8-4122-9e06-0c84c2977b7d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.873746 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.872751 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6r8x\" (UniqueName: \"kubernetes.io/projected/4396d907-2db8-4122-9e06-0c84c2977b7d-kube-api-access-l6r8x\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.873746 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.872782 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4162058-cbae-40d7-b6c2-e2a3133fd372-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-27chd\" (UID: \"f4162058-cbae-40d7-b6c2-e2a3133fd372\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd" Jul 29 14:36:30.873746 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.872809 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64qg2\" (UniqueName: \"kubernetes.io/projected/f4162058-cbae-40d7-b6c2-e2a3133fd372-kube-api-access-64qg2\") pod \"openshift-state-metrics-b8f49f5c6-27chd\" (UID: \"f4162058-cbae-40d7-b6c2-e2a3133fd372\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd" Jul 29 14:36:30.873746 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.872837 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4162058-cbae-40d7-b6c2-e2a3133fd372-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-27chd\" (UID: \"f4162058-cbae-40d7-b6c2-e2a3133fd372\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd" Jul 29 14:36:30.873746 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.872859 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4396d907-2db8-4122-9e06-0c84c2977b7d-sys\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.873746 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.872884 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4396d907-2db8-4122-9e06-0c84c2977b7d-node-exporter-textfile\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.873746 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.872923 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4396d907-2db8-4122-9e06-0c84c2977b7d-root\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.873746 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.872946 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4396d907-2db8-4122-9e06-0c84c2977b7d-node-exporter-tls\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.873746 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.872976 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4396d907-2db8-4122-9e06-0c84c2977b7d-node-exporter-wtmp\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.873746 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.873003 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4396d907-2db8-4122-9e06-0c84c2977b7d-node-exporter-accelerators-collector-config\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.873746 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.873032 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4396d907-2db8-4122-9e06-0c84c2977b7d-metrics-client-ca\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.873746 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.873065 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4162058-cbae-40d7-b6c2-e2a3133fd372-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-27chd\" (UID: \"f4162058-cbae-40d7-b6c2-e2a3133fd372\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd" Jul 29 14:36:30.873746 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.873552 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4396d907-2db8-4122-9e06-0c84c2977b7d-sys\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.876402 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:36:30.874911 2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 29 14:36:30.876402 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:36:30.874980 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4396d907-2db8-4122-9e06-0c84c2977b7d-node-exporter-tls podName:4396d907-2db8-4122-9e06-0c84c2977b7d nodeName:}" failed. No retries permitted until 2026-07-29 14:36:31.374957819 +0000 UTC m=+68.103106991 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4396d907-2db8-4122-9e06-0c84c2977b7d-node-exporter-tls") pod "node-exporter-kqphh" (UID: "4396d907-2db8-4122-9e06-0c84c2977b7d") : secret "node-exporter-tls" not found Jul 29 14:36:30.876402 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.875421 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4396d907-2db8-4122-9e06-0c84c2977b7d-node-exporter-textfile\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.876402 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.875481 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4396d907-2db8-4122-9e06-0c84c2977b7d-root\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.876402 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.876251 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4396d907-2db8-4122-9e06-0c84c2977b7d-node-exporter-accelerators-collector-config\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.876402 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.876284 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4162058-cbae-40d7-b6c2-e2a3133fd372-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-27chd\" (UID: \"f4162058-cbae-40d7-b6c2-e2a3133fd372\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd" Jul 29 14:36:30.876402 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.876374 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4396d907-2db8-4122-9e06-0c84c2977b7d-node-exporter-wtmp\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.876913 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.876865 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4396d907-2db8-4122-9e06-0c84c2977b7d-metrics-client-ca\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.879699 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.879646 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f4162058-cbae-40d7-b6c2-e2a3133fd372-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-27chd\" (UID: \"f4162058-cbae-40d7-b6c2-e2a3133fd372\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd" Jul 29 14:36:30.880829 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.880798 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4162058-cbae-40d7-b6c2-e2a3133fd372-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-27chd\" (UID: \"f4162058-cbae-40d7-b6c2-e2a3133fd372\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd" Jul 29 14:36:30.882078 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.882037 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4396d907-2db8-4122-9e06-0c84c2977b7d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.884445 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.884421 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64qg2\" (UniqueName: \"kubernetes.io/projected/f4162058-cbae-40d7-b6c2-e2a3133fd372-kube-api-access-64qg2\") pod \"openshift-state-metrics-b8f49f5c6-27chd\" (UID: \"f4162058-cbae-40d7-b6c2-e2a3133fd372\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd" Jul 29 14:36:30.886553 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.886513 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6r8x\" (UniqueName: \"kubernetes.io/projected/4396d907-2db8-4122-9e06-0c84c2977b7d-kube-api-access-l6r8x\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:30.905486 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:30.905422 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:31.013295 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.013216 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:31.024345 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.024284 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd" Jul 29 14:36:31.039414 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.038629 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hm2lz" Jul 29 14:36:31.106859 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.106831 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:31.209070 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.209024 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:31.228134 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.228108 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd"] Jul 29 14:36:31.237919 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:36:31.237820 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4162058_cbae_40d7_b6c2_e2a3133fd372.slice/crio-254455ab37817dbf6f7ba13a06b8034122ad4fc0611cc99bcccb1084782158ba WatchSource:0}: Error finding container 254455ab37817dbf6f7ba13a06b8034122ad4fc0611cc99bcccb1084782158ba: Status 404 returned error can't find the container with id 254455ab37817dbf6f7ba13a06b8034122ad4fc0611cc99bcccb1084782158ba Jul 29 14:36:31.385872 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.385846 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4396d907-2db8-4122-9e06-0c84c2977b7d-node-exporter-tls\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:31.388963 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.388510 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4396d907-2db8-4122-9e06-0c84c2977b7d-node-exporter-tls\") pod \"node-exporter-kqphh\" (UID: \"4396d907-2db8-4122-9e06-0c84c2977b7d\") " pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:31.615229 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.615032 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:31.632794 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.632764 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kqphh" Jul 29 14:36:31.655879 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:36:31.655836 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4396d907_2db8_4122_9e06_0c84c2977b7d.slice/crio-5c9a0c46d42ff1a688166a0f95779cbd77c78772bd00dd80a3245788a249a952 WatchSource:0}: Error finding container 5c9a0c46d42ff1a688166a0f95779cbd77c78772bd00dd80a3245788a249a952: Status 404 returned error can't find the container with id 5c9a0c46d42ff1a688166a0f95779cbd77c78772bd00dd80a3245788a249a952 Jul 29 14:36:31.715903 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.715828 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:31.779455 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.779329 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 29 14:36:31.783003 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.782413 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.784675 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.784652 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5hvmz\"" Jul 29 14:36:31.784784 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.784688 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 29 14:36:31.785012 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.784993 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 29 14:36:31.785198 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.785177 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 29 14:36:31.785308 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.785292 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 29 14:36:31.785428 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.785413 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 29 14:36:31.785657 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.785624 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 29 14:36:31.785657 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.785643 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 29 14:36:31.785924 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.785909 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 29 14:36:31.785986 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.785925 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 29 14:36:31.803189 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.803167 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 29 14:36:31.809690 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.809587 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:31.890457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.890392 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.890457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.890435 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48b307f0-dd22-4daf-b297-9b8edd927821-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.890890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.890476 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48b307f0-dd22-4daf-b297-9b8edd927821-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.890890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.890510 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/48b307f0-dd22-4daf-b297-9b8edd927821-tls-assets\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.890890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.890535 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7vrj\" (UniqueName: \"kubernetes.io/projected/48b307f0-dd22-4daf-b297-9b8edd927821-kube-api-access-s7vrj\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.890890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.890586 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.890890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.890624 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.890890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.890658 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.890890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.890686 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/48b307f0-dd22-4daf-b297-9b8edd927821-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.890890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.890711 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.890890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.890764 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-web-config\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.890890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.890809 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-config-volume\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.890890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.890834 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/48b307f0-dd22-4daf-b297-9b8edd927821-config-out\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.934778 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.934751 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:31.991987 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.991959 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.992130 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.992001 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48b307f0-dd22-4daf-b297-9b8edd927821-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.992130 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.992035 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48b307f0-dd22-4daf-b297-9b8edd927821-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.992130 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.992080 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/48b307f0-dd22-4daf-b297-9b8edd927821-tls-assets\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.992130 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.992106 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7vrj\" (UniqueName: \"kubernetes.io/projected/48b307f0-dd22-4daf-b297-9b8edd927821-kube-api-access-s7vrj\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.992333 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.992141 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.992333 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.992166 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.992333 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.992199 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.992333 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.992226 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/48b307f0-dd22-4daf-b297-9b8edd927821-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.992333 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.992262 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.992333 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.992288 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-web-config\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.992660 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.992335 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-config-volume\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.992660 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.992360 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/48b307f0-dd22-4daf-b297-9b8edd927821-config-out\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.996673 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.994017 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48b307f0-dd22-4daf-b297-9b8edd927821-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.996673 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.994303 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/48b307f0-dd22-4daf-b297-9b8edd927821-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:31.996673 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:31.995747 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48b307f0-dd22-4daf-b297-9b8edd927821-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:32.000784 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:32.000738 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:32.000873 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:32.000803 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/48b307f0-dd22-4daf-b297-9b8edd927821-tls-assets\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:32.001468 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:32.001438 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:32.003023 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:32.002517 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-config-volume\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:32.003023 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:32.002973 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:32.003639 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:32.003552 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-web-config\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:32.005075 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:32.004998 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:32.005427 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:32.005369 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/48b307f0-dd22-4daf-b297-9b8edd927821-config-out\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:32.006111 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:32.006073 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7vrj\" (UniqueName: \"kubernetes.io/projected/48b307f0-dd22-4daf-b297-9b8edd927821-kube-api-access-s7vrj\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:32.008274 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:32.008196 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:32.101173 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:32.096545 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:36:32.105016 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:32.103701 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kqphh" event={"ID":"4396d907-2db8-4122-9e06-0c84c2977b7d","Type":"ContainerStarted","Data":"5c9a0c46d42ff1a688166a0f95779cbd77c78772bd00dd80a3245788a249a952"} Jul 29 14:36:32.112475 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:32.112438 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd" event={"ID":"f4162058-cbae-40d7-b6c2-e2a3133fd372","Type":"ContainerStarted","Data":"4e900a7a34933d1e2960958b5404239c82dd685783aad059393eb2a1045977a5"} Jul 29 14:36:32.112579 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:32.112499 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd" event={"ID":"f4162058-cbae-40d7-b6c2-e2a3133fd372","Type":"ContainerStarted","Data":"04c3b2f6b0ef84784bc159d8033b6e6650333fb31f2520c5e4e5708bf23a3c0e"} Jul 29 14:36:32.112579 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:32.112528 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd" event={"ID":"f4162058-cbae-40d7-b6c2-e2a3133fd372","Type":"ContainerStarted","Data":"254455ab37817dbf6f7ba13a06b8034122ad4fc0611cc99bcccb1084782158ba"} Jul 29 14:36:32.116199 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:32.116175 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fmd7q" event={"ID":"1e27a282-f2d3-4aa9-81d3-468c1da0ab8e","Type":"ContainerStarted","Data":"4f887682b226ba6c5732bf106cc8d3b35c1b9fd3f585c73c20ff23b69d1273ac"} Jul 29 14:36:32.116294 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:32.116224 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fmd7q" event={"ID":"1e27a282-f2d3-4aa9-81d3-468c1da0ab8e","Type":"ContainerStarted","Data":"b4a9c039588a15e1a7b2f472cf5c75863ef648df764145e98b6779553dbc9109"} Jul 29 14:36:32.138133 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:32.137717 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fmd7q" podStartSLOduration=67.917246309 podStartE2EDuration="1m9.137698395s" totalImagesPullingTime="1.220452086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:35:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:36:32.136412953 +0000 UTC m=+68.864562142" watchObservedRunningTime="2026-07-29 14:36:32.137698395 +0000 UTC m=+68.865847585" Jul 29 14:36:32.247361 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:32.247289 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:32.279938 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:32.279878 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 29 14:36:32.348318 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:32.348232 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:33.457412 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:33.457377 2580 patch_prober.go:28] interesting pod/image-registry-847ddf4c6-zlbq6 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 29 14:36:33.457793 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:33.457426 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" podUID="814fcac9-b1b5-4d9f-9146-d06195946b47" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 14:36:33.728528 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:33.728450 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:33.837801 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:33.837760 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:34.085517 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:34.085478 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:34.322107 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:34.322046 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:35.012195 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:35.012167 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-847ddf4c6-zlbq6" Jul 29 14:36:35.207022 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:35.206970 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:35.306890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:35.306815 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:35.416197 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:35.416162 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:35.476676 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:35.476649 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-ttcjn"] Jul 29 14:36:35.478877 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:35.478856 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-ttcjn" Jul 29 14:36:35.481127 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:35.481084 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 29 14:36:35.481127 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:35.481132 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2fbgj\"" Jul 29 14:36:35.489699 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:35.489677 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-ttcjn"] Jul 29 14:36:35.624332 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:35.624252 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/716f191c-64ee-4690-b427-76f378fd3484-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-ttcjn\" (UID: \"716f191c-64ee-4690-b427-76f378fd3484\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-ttcjn" Jul 29 14:36:35.624332 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:35.624322 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnl6l\" (UniqueName: \"kubernetes.io/projected/716f191c-64ee-4690-b427-76f378fd3484-kube-api-access-dnl6l\") pod \"monitoring-plugin-68b7fc9574-ttcjn\" (UID: \"716f191c-64ee-4690-b427-76f378fd3484\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-ttcjn" Jul 29 14:36:35.725732 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:35.725701 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnl6l\" (UniqueName: \"kubernetes.io/projected/716f191c-64ee-4690-b427-76f378fd3484-kube-api-access-dnl6l\") pod \"monitoring-plugin-68b7fc9574-ttcjn\" (UID: \"716f191c-64ee-4690-b427-76f378fd3484\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-ttcjn" Jul 29 14:36:35.725881 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:35.725792 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/716f191c-64ee-4690-b427-76f378fd3484-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-ttcjn\" (UID: \"716f191c-64ee-4690-b427-76f378fd3484\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-ttcjn" Jul 29 14:36:35.728602 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:35.728573 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/716f191c-64ee-4690-b427-76f378fd3484-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-ttcjn\" (UID: \"716f191c-64ee-4690-b427-76f378fd3484\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-ttcjn" Jul 29 14:36:35.734144 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:35.734122 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnl6l\" (UniqueName: \"kubernetes.io/projected/716f191c-64ee-4690-b427-76f378fd3484-kube-api-access-dnl6l\") pod \"monitoring-plugin-68b7fc9574-ttcjn\" (UID: \"716f191c-64ee-4690-b427-76f378fd3484\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-ttcjn" Jul 29 14:36:35.790341 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:35.790318 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-ttcjn" Jul 29 14:36:36.509216 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:36.509178 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:36.564203 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:36.564163 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:37.108806 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:37.108773 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:37.172224 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:37.172191 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:37.763612 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:37.763573 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:37.864399 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:37.864367 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:38.148516 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:38.148442 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:38.249060 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:38.249031 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:39.521509 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:36:39.521473 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48b307f0_dd22_4daf_b297_9b8edd927821.slice/crio-15e69cc51c741f6873e996ae1428e88e1b2b2a3db190c1b277d14ab02cd7975e WatchSource:0}: Error finding container 15e69cc51c741f6873e996ae1428e88e1b2b2a3db190c1b277d14ab02cd7975e: Status 404 returned error can't find the container with id 15e69cc51c741f6873e996ae1428e88e1b2b2a3db190c1b277d14ab02cd7975e Jul 29 14:36:40.148839 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:40.148807 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"48b307f0-dd22-4daf-b297-9b8edd927821","Type":"ContainerStarted","Data":"15e69cc51c741f6873e996ae1428e88e1b2b2a3db190c1b277d14ab02cd7975e"} Jul 29 14:36:41.278933 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.278898 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-dd5f88445-z44jz"] Jul 29 14:36:41.288228 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.288206 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.293501 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.293478 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 29 14:36:41.293620 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.293507 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 29 14:36:41.293620 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.293543 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fxbjz\"" Jul 29 14:36:41.293620 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.293509 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 29 14:36:41.293620 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.293562 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 29 14:36:41.293620 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.293489 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 29 14:36:41.302185 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.302151 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dd5f88445-z44jz"] Jul 29 14:36:41.306303 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.306279 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 29 14:36:41.371200 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.371170 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5007118f-0fad-4036-b794-e942e6d52d1a-console-serving-cert\") pod \"console-dd5f88445-z44jz\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.371342 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.371233 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-oauth-serving-cert\") pod \"console-dd5f88445-z44jz\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.371342 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.371258 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-service-ca\") pod \"console-dd5f88445-z44jz\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.371342 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.371298 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-trusted-ca-bundle\") pod \"console-dd5f88445-z44jz\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.371342 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.371324 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtg4z\" (UniqueName: \"kubernetes.io/projected/5007118f-0fad-4036-b794-e942e6d52d1a-kube-api-access-qtg4z\") pod \"console-dd5f88445-z44jz\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.371561 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.371366 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5007118f-0fad-4036-b794-e942e6d52d1a-console-oauth-config\") pod \"console-dd5f88445-z44jz\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.371561 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.371407 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-console-config\") pod \"console-dd5f88445-z44jz\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.471908 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.471879 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtg4z\" (UniqueName: \"kubernetes.io/projected/5007118f-0fad-4036-b794-e942e6d52d1a-kube-api-access-qtg4z\") pod \"console-dd5f88445-z44jz\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.472033 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.471930 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5007118f-0fad-4036-b794-e942e6d52d1a-console-oauth-config\") pod \"console-dd5f88445-z44jz\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.472033 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.471948 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-console-config\") pod \"console-dd5f88445-z44jz\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.472161 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.472137 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5007118f-0fad-4036-b794-e942e6d52d1a-console-serving-cert\") pod \"console-dd5f88445-z44jz\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.472236 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.472221 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-oauth-serving-cert\") pod \"console-dd5f88445-z44jz\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.472288 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.472256 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-service-ca\") pod \"console-dd5f88445-z44jz\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.472338 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.472289 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-trusted-ca-bundle\") pod \"console-dd5f88445-z44jz\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.472748 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.472685 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-console-config\") pod \"console-dd5f88445-z44jz\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.472956 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.472932 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-oauth-serving-cert\") pod \"console-dd5f88445-z44jz\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.473057 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.472967 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-service-ca\") pod \"console-dd5f88445-z44jz\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.473146 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.473122 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-trusted-ca-bundle\") pod \"console-dd5f88445-z44jz\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.474551 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.474534 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5007118f-0fad-4036-b794-e942e6d52d1a-console-oauth-config\") pod \"console-dd5f88445-z44jz\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.474874 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.474857 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5007118f-0fad-4036-b794-e942e6d52d1a-console-serving-cert\") pod \"console-dd5f88445-z44jz\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.482776 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.482758 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtg4z\" (UniqueName: \"kubernetes.io/projected/5007118f-0fad-4036-b794-e942e6d52d1a-kube-api-access-qtg4z\") pod \"console-dd5f88445-z44jz\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:41.597696 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:41.597636 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:43.316565 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:43.311150 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-ttcjn"] Jul 29 14:36:43.331118 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:36:43.330749 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod716f191c_64ee_4690_b427_76f378fd3484.slice/crio-6a7e2e6dccce6af1b16dd9ec1db1ea396148f14b9b971315e530608433310c72 WatchSource:0}: Error finding container 6a7e2e6dccce6af1b16dd9ec1db1ea396148f14b9b971315e530608433310c72: Status 404 returned error can't find the container with id 6a7e2e6dccce6af1b16dd9ec1db1ea396148f14b9b971315e530608433310c72 Jul 29 14:36:43.348760 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:43.348265 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dd5f88445-z44jz"] Jul 29 14:36:43.361967 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:43.361940 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:43.463481 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:43.463293 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:43.658775 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:36:43.658740 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5007118f_0fad_4036_b794_e942e6d52d1a.slice/crio-9b4c8841338d9870d6eaa423cf34933574d5c61d706d3ad45a8ee54ec7a15f89 WatchSource:0}: Error finding container 9b4c8841338d9870d6eaa423cf34933574d5c61d706d3ad45a8ee54ec7a15f89: Status 404 returned error can't find the container with id 9b4c8841338d9870d6eaa423cf34933574d5c61d706d3ad45a8ee54ec7a15f89 Jul 29 14:36:44.167148 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:44.167116 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tfbhf" event={"ID":"3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9","Type":"ContainerStarted","Data":"54da96f337e7e05510d3f69550d764045eed6ca6f0a13df5ec9aa08b1344dc44"} Jul 29 14:36:44.174336 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:44.173713 2580 generic.go:356] "Generic (PLEG): container finished" podID="48b307f0-dd22-4daf-b297-9b8edd927821" containerID="32a528beb823da80867e358a4bff51b181a2b590c1709087843f5f00cfb40c24" exitCode=0 Jul 29 14:36:44.174467 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:44.174437 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"48b307f0-dd22-4daf-b297-9b8edd927821","Type":"ContainerDied","Data":"32a528beb823da80867e358a4bff51b181a2b590c1709087843f5f00cfb40c24"} Jul 29 14:36:44.178704 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:44.178674 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-vwwj2" event={"ID":"16bddc73-bd54-4f98-9041-5b0848562b4a","Type":"ContainerStarted","Data":"dffb0252a3455a330ac4bcb567a871e0aafc8e7b93e8d8e1ae692deeca5e3363"} Jul 29 14:36:44.179138 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:44.179119 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-vwwj2" Jul 29 14:36:44.182574 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:44.182550 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vv7bl" event={"ID":"10c2f512-d00b-4f28-8957-4311e81257ec","Type":"ContainerStarted","Data":"aa41ad6c3137c094d8ac114f4c2bdd512cd7f873e1910cd5a6544cebc0149a41"} Jul 29 14:36:44.182677 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:44.182652 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:36:44.188748 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:44.188270 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-ttcjn" event={"ID":"716f191c-64ee-4690-b427-76f378fd3484","Type":"ContainerStarted","Data":"6a7e2e6dccce6af1b16dd9ec1db1ea396148f14b9b971315e530608433310c72"} Jul 29 14:36:44.192948 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:44.192801 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dd5f88445-z44jz" event={"ID":"5007118f-0fad-4036-b794-e942e6d52d1a","Type":"ContainerStarted","Data":"9b4c8841338d9870d6eaa423cf34933574d5c61d706d3ad45a8ee54ec7a15f89"} Jul 29 14:36:44.194592 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:44.194550 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tfbhf" podStartSLOduration=66.904137738 podStartE2EDuration="1m20.194534573s" totalImagesPullingTime="13.290396835s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 14:35:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:36:44.18522279 +0000 UTC m=+80.913371981" watchObservedRunningTime="2026-07-29 14:36:44.194534573 +0000 UTC m=+80.922683760" Jul 29 14:36:44.197470 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:44.197447 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-vwwj2" Jul 29 14:36:44.201163 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:44.200429 2580 generic.go:356] "Generic (PLEG): container finished" podID="4396d907-2db8-4122-9e06-0c84c2977b7d" containerID="0863b25167ac4b4cf40482ae2abdacb622b32884162defd32969a08a8b51e5ca" exitCode=0 Jul 29 14:36:44.201410 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:44.201339 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kqphh" event={"ID":"4396d907-2db8-4122-9e06-0c84c2977b7d","Type":"ContainerDied","Data":"0863b25167ac4b4cf40482ae2abdacb622b32884162defd32969a08a8b51e5ca"} Jul 29 14:36:44.206611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:44.206574 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-vwwj2" podStartSLOduration=6.279808202 podStartE2EDuration="29.206560063s" totalImagesPullingTime="22.926751861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:36:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:36:44.206097417 +0000 UTC m=+80.934246604" watchObservedRunningTime="2026-07-29 14:36:44.206560063 +0000 UTC m=+80.934709251" Jul 29 14:36:44.208653 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:44.208600 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd" event={"ID":"f4162058-cbae-40d7-b6c2-e2a3133fd372","Type":"ContainerStarted","Data":"5e919a700867513d67a9d1165131623184fb106379c04f3370fefc6cab7ff77b"} Jul 29 14:36:44.221042 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:44.220742 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vv7bl" podStartSLOduration=68.049004474 podStartE2EDuration="1m21.220710019s" totalImagesPullingTime="13.171705545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:35:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:36:44.220169027 +0000 UTC m=+80.948318234" watchObservedRunningTime="2026-07-29 14:36:44.220710019 +0000 UTC m=+80.948859205" Jul 29 14:36:44.343103 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:44.342343 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-27chd" podStartSLOduration=2.595684424 podStartE2EDuration="14.342320573s" totalImagesPullingTime="11.746636149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:36:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:36:44.341058856 +0000 UTC m=+81.069208066" watchObservedRunningTime="2026-07-29 14:36:44.342320573 +0000 UTC m=+81.070469766" Jul 29 14:36:45.192538 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:45.192349 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:45.219044 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:45.218134 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kqphh" event={"ID":"4396d907-2db8-4122-9e06-0c84c2977b7d","Type":"ContainerStarted","Data":"d729d748bfa57356fa64b1618858ef2cd827062c29f7f824b23e873c0fd08795"} Jul 29 14:36:45.219044 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:45.218192 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kqphh" event={"ID":"4396d907-2db8-4122-9e06-0c84c2977b7d","Type":"ContainerStarted","Data":"86d7091b92480a7da1c4e658e453e52153f9b70e4c2fd79d303cf1a5a385f83d"} Jul 29 14:36:45.237394 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:45.237062 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kqphh" podStartSLOduration=3.6770421620000002 podStartE2EDuration="15.237045402s" totalImagesPullingTime="11.48623516s" totalInitContainerRuntime="73.76808ms" isStatefulPod=true podCreationTimestamp="2026-07-29 14:36:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:36:45.236659005 +0000 UTC m=+81.964808242" watchObservedRunningTime="2026-07-29 14:36:45.237045402 +0000 UTC m=+81.965194591" Jul 29 14:36:45.293393 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:45.293333 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:47.224138 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:47.224097 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:47.324002 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:47.323957 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:48.233245 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:48.233220 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"48b307f0-dd22-4daf-b297-9b8edd927821","Type":"ContainerStarted","Data":"0cb48b4df24e2fea384386cf2c1401554bfe1c8c7622e37962fedc8c346c131c"} Jul 29 14:36:48.235091 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:48.235010 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-ttcjn" event={"ID":"716f191c-64ee-4690-b427-76f378fd3484","Type":"ContainerStarted","Data":"fb2b1bd368e4fefdd5c66d83803a019fa944d370d628c1118d4d8fb18c896a43"} Jul 29 14:36:48.236564 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:48.236542 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dd5f88445-z44jz" event={"ID":"5007118f-0fad-4036-b794-e942e6d52d1a","Type":"ContainerStarted","Data":"58e2e76d0a2ed8e6b6c875845964e162bdfb01b09c8f3ecfff1ecf95f1aa8667"} Jul 29 14:36:48.248636 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:48.248203 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-ttcjn" Jul 29 14:36:48.253606 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:48.253584 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-ttcjn" Jul 29 14:36:48.263814 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:48.263766 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-ttcjn" podStartSLOduration=8.590327704 podStartE2EDuration="13.263749132s" totalImagesPullingTime="4.673421428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:36:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:36:48.26302652 +0000 UTC m=+84.991175732" watchObservedRunningTime="2026-07-29 14:36:48.263749132 +0000 UTC m=+84.991898316" Jul 29 14:36:48.286205 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:48.286155 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-dd5f88445-z44jz" podStartSLOduration=2.902692729 podStartE2EDuration="7.286139462s" totalImagesPullingTime="4.383446733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:36:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:36:48.285537641 +0000 UTC m=+85.013686854" watchObservedRunningTime="2026-07-29 14:36:48.286139462 +0000 UTC m=+85.014288655" Jul 29 14:36:49.243826 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:49.243792 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"48b307f0-dd22-4daf-b297-9b8edd927821","Type":"ContainerStarted","Data":"34905cef389b268d13a2d51cb0ea9d976845558ab33bc6c96659384916446203"} Jul 29 14:36:49.244226 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:49.243876 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"48b307f0-dd22-4daf-b297-9b8edd927821","Type":"ContainerStarted","Data":"e5f220c01510984cfc55b0408efbddadaea1b68eee58ecb498da0a232b7f04b3"} Jul 29 14:36:49.244226 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:49.243909 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"48b307f0-dd22-4daf-b297-9b8edd927821","Type":"ContainerStarted","Data":"526e24694b54afbfc20f344231ae1d8419373fd0e878f163fcc02c9cfc636f44"} Jul 29 14:36:49.244226 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:49.243951 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"48b307f0-dd22-4daf-b297-9b8edd927821","Type":"ContainerStarted","Data":"1edab304cf606337d47ef4073865b10e1af68b32763f760b451c4c43917ba836"} Jul 29 14:36:51.256741 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:51.256689 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"48b307f0-dd22-4daf-b297-9b8edd927821","Type":"ContainerStarted","Data":"2944a2c30b7bb718f240ee796a05bf159dd86dbb974865705c4fb2157bc005cf"} Jul 29 14:36:51.287987 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:51.287929 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=10.667458721 podStartE2EDuration="20.287901966s" totalImagesPullingTime="9.6016189s" totalInitContainerRuntime="18.824345ms" isStatefulPod=false podCreationTimestamp="2026-07-29 14:36:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:36:51.286802559 +0000 UTC m=+88.014951771" watchObservedRunningTime="2026-07-29 14:36:51.287901966 +0000 UTC m=+88.016051155" Jul 29 14:36:51.597850 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:51.597771 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:51.597850 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:51.597838 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:51.603298 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:51.603272 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:52.264147 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:52.264121 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:36:52.449993 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:52.449958 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:52.551339 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:52.551260 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:53.513820 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:53.513785 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:36:53.592117 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:36:53.592081 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:37:01.423769 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:01.423689 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:37:01.526066 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:01.526029 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:37:01.627563 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:01.627533 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:37:01.687078 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:01.687021 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:37:15.221841 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:15.221806 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vv7bl" Jul 29 14:37:49.999396 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:49.999357 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:37:50.099156 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:50.099121 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:37:51.055838 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:51.055803 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 29 14:37:51.056358 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:51.056119 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="alertmanager" containerID="cri-o://0cb48b4df24e2fea384386cf2c1401554bfe1c8c7622e37962fedc8c346c131c" gracePeriod=120 Jul 29 14:37:51.056358 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:51.056149 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="prom-label-proxy" containerID="cri-o://2944a2c30b7bb718f240ee796a05bf159dd86dbb974865705c4fb2157bc005cf" gracePeriod=120 Jul 29 14:37:51.056358 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:51.056163 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="kube-rbac-proxy-metric" containerID="cri-o://34905cef389b268d13a2d51cb0ea9d976845558ab33bc6c96659384916446203" gracePeriod=120 Jul 29 14:37:51.056358 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:51.056218 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="config-reloader" containerID="cri-o://1edab304cf606337d47ef4073865b10e1af68b32763f760b451c4c43917ba836" gracePeriod=120 Jul 29 14:37:51.056358 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:51.056288 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="kube-rbac-proxy-web" containerID="cri-o://526e24694b54afbfc20f344231ae1d8419373fd0e878f163fcc02c9cfc636f44" gracePeriod=120 Jul 29 14:37:51.056588 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:51.056131 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="kube-rbac-proxy" containerID="cri-o://e5f220c01510984cfc55b0408efbddadaea1b68eee58ecb498da0a232b7f04b3" gracePeriod=120 Jul 29 14:37:51.478473 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:51.478415 2580 generic.go:356] "Generic (PLEG): container finished" podID="48b307f0-dd22-4daf-b297-9b8edd927821" containerID="2944a2c30b7bb718f240ee796a05bf159dd86dbb974865705c4fb2157bc005cf" exitCode=0 Jul 29 14:37:51.478473 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:51.478436 2580 generic.go:356] "Generic (PLEG): container finished" podID="48b307f0-dd22-4daf-b297-9b8edd927821" containerID="e5f220c01510984cfc55b0408efbddadaea1b68eee58ecb498da0a232b7f04b3" exitCode=0 Jul 29 14:37:51.478473 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:51.478444 2580 generic.go:356] "Generic (PLEG): container finished" podID="48b307f0-dd22-4daf-b297-9b8edd927821" containerID="1edab304cf606337d47ef4073865b10e1af68b32763f760b451c4c43917ba836" exitCode=0 Jul 29 14:37:51.478473 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:51.478450 2580 generic.go:356] "Generic (PLEG): container finished" podID="48b307f0-dd22-4daf-b297-9b8edd927821" containerID="0cb48b4df24e2fea384386cf2c1401554bfe1c8c7622e37962fedc8c346c131c" exitCode=0 Jul 29 14:37:51.478659 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:51.478489 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"48b307f0-dd22-4daf-b297-9b8edd927821","Type":"ContainerDied","Data":"2944a2c30b7bb718f240ee796a05bf159dd86dbb974865705c4fb2157bc005cf"} Jul 29 14:37:51.478659 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:51.478554 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"48b307f0-dd22-4daf-b297-9b8edd927821","Type":"ContainerDied","Data":"e5f220c01510984cfc55b0408efbddadaea1b68eee58ecb498da0a232b7f04b3"} Jul 29 14:37:51.478659 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:51.478581 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"48b307f0-dd22-4daf-b297-9b8edd927821","Type":"ContainerDied","Data":"1edab304cf606337d47ef4073865b10e1af68b32763f760b451c4c43917ba836"} Jul 29 14:37:51.478659 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:51.478606 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"48b307f0-dd22-4daf-b297-9b8edd927821","Type":"ContainerDied","Data":"0cb48b4df24e2fea384386cf2c1401554bfe1c8c7622e37962fedc8c346c131c"} Jul 29 14:37:52.291097 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.291073 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.399438 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.399379 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-kube-rbac-proxy-metric\") pod \"48b307f0-dd22-4daf-b297-9b8edd927821\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " Jul 29 14:37:52.399438 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.399418 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48b307f0-dd22-4daf-b297-9b8edd927821-alertmanager-trusted-ca-bundle\") pod \"48b307f0-dd22-4daf-b297-9b8edd927821\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " Jul 29 14:37:52.399617 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.399441 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/48b307f0-dd22-4daf-b297-9b8edd927821-config-out\") pod \"48b307f0-dd22-4daf-b297-9b8edd927821\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " Jul 29 14:37:52.399617 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.399467 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7vrj\" (UniqueName: \"kubernetes.io/projected/48b307f0-dd22-4daf-b297-9b8edd927821-kube-api-access-s7vrj\") pod \"48b307f0-dd22-4daf-b297-9b8edd927821\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " Jul 29 14:37:52.399617 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.399485 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-main-tls\") pod \"48b307f0-dd22-4daf-b297-9b8edd927821\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " Jul 29 14:37:52.399617 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.399507 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-config-volume\") pod \"48b307f0-dd22-4daf-b297-9b8edd927821\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " Jul 29 14:37:52.399617 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.399534 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48b307f0-dd22-4daf-b297-9b8edd927821-metrics-client-ca\") pod \"48b307f0-dd22-4daf-b297-9b8edd927821\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " Jul 29 14:37:52.399617 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.399561 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-web-config\") pod \"48b307f0-dd22-4daf-b297-9b8edd927821\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " Jul 29 14:37:52.399617 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.399614 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-kube-rbac-proxy-web\") pod \"48b307f0-dd22-4daf-b297-9b8edd927821\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " Jul 29 14:37:52.399999 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.399638 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/48b307f0-dd22-4daf-b297-9b8edd927821-alertmanager-main-db\") pod \"48b307f0-dd22-4daf-b297-9b8edd927821\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " Jul 29 14:37:52.399999 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.399665 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-cluster-tls-config\") pod \"48b307f0-dd22-4daf-b297-9b8edd927821\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " Jul 29 14:37:52.399999 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.399701 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/48b307f0-dd22-4daf-b297-9b8edd927821-tls-assets\") pod \"48b307f0-dd22-4daf-b297-9b8edd927821\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " Jul 29 14:37:52.399999 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.399764 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-kube-rbac-proxy\") pod \"48b307f0-dd22-4daf-b297-9b8edd927821\" (UID: \"48b307f0-dd22-4daf-b297-9b8edd927821\") " Jul 29 14:37:52.399999 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.399858 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48b307f0-dd22-4daf-b297-9b8edd927821-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "48b307f0-dd22-4daf-b297-9b8edd927821" (UID: "48b307f0-dd22-4daf-b297-9b8edd927821"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 14:37:52.400233 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.399999 2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48b307f0-dd22-4daf-b297-9b8edd927821-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:37:52.400471 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.400374 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48b307f0-dd22-4daf-b297-9b8edd927821-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "48b307f0-dd22-4daf-b297-9b8edd927821" (UID: "48b307f0-dd22-4daf-b297-9b8edd927821"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:37:52.400471 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.400455 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48b307f0-dd22-4daf-b297-9b8edd927821-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "48b307f0-dd22-4daf-b297-9b8edd927821" (UID: "48b307f0-dd22-4daf-b297-9b8edd927821"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 14:37:52.403424 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.403375 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "48b307f0-dd22-4daf-b297-9b8edd927821" (UID: "48b307f0-dd22-4daf-b297-9b8edd927821"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 14:37:52.403424 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.403383 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48b307f0-dd22-4daf-b297-9b8edd927821-config-out" (OuterVolumeSpecName: "config-out") pod "48b307f0-dd22-4daf-b297-9b8edd927821" (UID: "48b307f0-dd22-4daf-b297-9b8edd927821"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:37:52.403424 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.403392 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48b307f0-dd22-4daf-b297-9b8edd927821-kube-api-access-s7vrj" (OuterVolumeSpecName: "kube-api-access-s7vrj") pod "48b307f0-dd22-4daf-b297-9b8edd927821" (UID: "48b307f0-dd22-4daf-b297-9b8edd927821"). InnerVolumeSpecName "kube-api-access-s7vrj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 14:37:52.403860 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.403821 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48b307f0-dd22-4daf-b297-9b8edd927821-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "48b307f0-dd22-4daf-b297-9b8edd927821" (UID: "48b307f0-dd22-4daf-b297-9b8edd927821"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 14:37:52.404004 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.403974 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "48b307f0-dd22-4daf-b297-9b8edd927821" (UID: "48b307f0-dd22-4daf-b297-9b8edd927821"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 14:37:52.404102 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.404022 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "48b307f0-dd22-4daf-b297-9b8edd927821" (UID: "48b307f0-dd22-4daf-b297-9b8edd927821"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 14:37:52.404142 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.404115 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "48b307f0-dd22-4daf-b297-9b8edd927821" (UID: "48b307f0-dd22-4daf-b297-9b8edd927821"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 14:37:52.404199 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.404177 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-config-volume" (OuterVolumeSpecName: "config-volume") pod "48b307f0-dd22-4daf-b297-9b8edd927821" (UID: "48b307f0-dd22-4daf-b297-9b8edd927821"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 14:37:52.406687 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.406470 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "48b307f0-dd22-4daf-b297-9b8edd927821" (UID: "48b307f0-dd22-4daf-b297-9b8edd927821"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 14:37:52.412546 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.412523 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-web-config" (OuterVolumeSpecName: "web-config") pod "48b307f0-dd22-4daf-b297-9b8edd927821" (UID: "48b307f0-dd22-4daf-b297-9b8edd927821"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 14:37:52.484914 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.484891 2580 generic.go:356] "Generic (PLEG): container finished" podID="48b307f0-dd22-4daf-b297-9b8edd927821" containerID="34905cef389b268d13a2d51cb0ea9d976845558ab33bc6c96659384916446203" exitCode=0 Jul 29 14:37:52.484914 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.484911 2580 generic.go:356] "Generic (PLEG): container finished" podID="48b307f0-dd22-4daf-b297-9b8edd927821" containerID="526e24694b54afbfc20f344231ae1d8419373fd0e878f163fcc02c9cfc636f44" exitCode=0 Jul 29 14:37:52.485029 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.484951 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"48b307f0-dd22-4daf-b297-9b8edd927821","Type":"ContainerDied","Data":"34905cef389b268d13a2d51cb0ea9d976845558ab33bc6c96659384916446203"} Jul 29 14:37:52.485029 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.484968 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.485029 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.485008 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"48b307f0-dd22-4daf-b297-9b8edd927821","Type":"ContainerDied","Data":"526e24694b54afbfc20f344231ae1d8419373fd0e878f163fcc02c9cfc636f44"} Jul 29 14:37:52.485117 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.485033 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"48b307f0-dd22-4daf-b297-9b8edd927821","Type":"ContainerDied","Data":"15e69cc51c741f6873e996ae1428e88e1b2b2a3db190c1b277d14ab02cd7975e"} Jul 29 14:37:52.485117 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.485061 2580 scope.go:117] "RemoveContainer" containerID="2944a2c30b7bb718f240ee796a05bf159dd86dbb974865705c4fb2157bc005cf" Jul 29 14:37:52.492447 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.492428 2580 scope.go:117] "RemoveContainer" containerID="34905cef389b268d13a2d51cb0ea9d976845558ab33bc6c96659384916446203" Jul 29 14:37:52.498632 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.498611 2580 scope.go:117] "RemoveContainer" containerID="e5f220c01510984cfc55b0408efbddadaea1b68eee58ecb498da0a232b7f04b3" Jul 29 14:37:52.500739 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.500702 2580 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-cluster-tls-config\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:37:52.500810 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.500792 2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/48b307f0-dd22-4daf-b297-9b8edd927821-tls-assets\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:37:52.500848 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.500809 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:37:52.500848 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.500825 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:37:52.500848 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.500838 2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/48b307f0-dd22-4daf-b297-9b8edd927821-config-out\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:37:52.500934 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.500852 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7vrj\" (UniqueName: \"kubernetes.io/projected/48b307f0-dd22-4daf-b297-9b8edd927821-kube-api-access-s7vrj\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:37:52.500934 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.500866 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-main-tls\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:37:52.500934 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.500878 2580 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:37:52.500934 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.500891 2580 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48b307f0-dd22-4daf-b297-9b8edd927821-metrics-client-ca\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:37:52.500934 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.500903 2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-web-config\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:37:52.500934 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.500915 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/48b307f0-dd22-4daf-b297-9b8edd927821-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:37:52.500934 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.500927 2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/48b307f0-dd22-4daf-b297-9b8edd927821-alertmanager-main-db\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:37:52.505155 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.505133 2580 scope.go:117] "RemoveContainer" containerID="526e24694b54afbfc20f344231ae1d8419373fd0e878f163fcc02c9cfc636f44" Jul 29 14:37:52.505851 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.505833 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 29 14:37:52.509812 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.509792 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 29 14:37:52.512044 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.512019 2580 scope.go:117] "RemoveContainer" containerID="1edab304cf606337d47ef4073865b10e1af68b32763f760b451c4c43917ba836" Jul 29 14:37:52.517960 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.517946 2580 scope.go:117] "RemoveContainer" containerID="0cb48b4df24e2fea384386cf2c1401554bfe1c8c7622e37962fedc8c346c131c" Jul 29 14:37:52.523989 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.523976 2580 scope.go:117] "RemoveContainer" containerID="32a528beb823da80867e358a4bff51b181a2b590c1709087843f5f00cfb40c24" Jul 29 14:37:52.529939 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.529905 2580 scope.go:117] "RemoveContainer" containerID="2944a2c30b7bb718f240ee796a05bf159dd86dbb974865705c4fb2157bc005cf" Jul 29 14:37:52.530182 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:37:52.530163 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2944a2c30b7bb718f240ee796a05bf159dd86dbb974865705c4fb2157bc005cf\": container with ID starting with 2944a2c30b7bb718f240ee796a05bf159dd86dbb974865705c4fb2157bc005cf not found: ID does not exist" containerID="2944a2c30b7bb718f240ee796a05bf159dd86dbb974865705c4fb2157bc005cf" Jul 29 14:37:52.530248 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.530188 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2944a2c30b7bb718f240ee796a05bf159dd86dbb974865705c4fb2157bc005cf"} err="failed to get container status \"2944a2c30b7bb718f240ee796a05bf159dd86dbb974865705c4fb2157bc005cf\": rpc error: code = NotFound desc = could not find container \"2944a2c30b7bb718f240ee796a05bf159dd86dbb974865705c4fb2157bc005cf\": container with ID starting with 2944a2c30b7bb718f240ee796a05bf159dd86dbb974865705c4fb2157bc005cf not found: ID does not exist" Jul 29 14:37:52.530248 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.530215 2580 scope.go:117] "RemoveContainer" containerID="34905cef389b268d13a2d51cb0ea9d976845558ab33bc6c96659384916446203" Jul 29 14:37:52.530457 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:37:52.530441 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34905cef389b268d13a2d51cb0ea9d976845558ab33bc6c96659384916446203\": container with ID starting with 34905cef389b268d13a2d51cb0ea9d976845558ab33bc6c96659384916446203 not found: ID does not exist" containerID="34905cef389b268d13a2d51cb0ea9d976845558ab33bc6c96659384916446203" Jul 29 14:37:52.530519 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.530477 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34905cef389b268d13a2d51cb0ea9d976845558ab33bc6c96659384916446203"} err="failed to get container status \"34905cef389b268d13a2d51cb0ea9d976845558ab33bc6c96659384916446203\": rpc error: code = NotFound desc = could not find container \"34905cef389b268d13a2d51cb0ea9d976845558ab33bc6c96659384916446203\": container with ID starting with 34905cef389b268d13a2d51cb0ea9d976845558ab33bc6c96659384916446203 not found: ID does not exist" Jul 29 14:37:52.530519 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.530499 2580 scope.go:117] "RemoveContainer" containerID="e5f220c01510984cfc55b0408efbddadaea1b68eee58ecb498da0a232b7f04b3" Jul 29 14:37:52.530757 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:37:52.530743 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5f220c01510984cfc55b0408efbddadaea1b68eee58ecb498da0a232b7f04b3\": container with ID starting with e5f220c01510984cfc55b0408efbddadaea1b68eee58ecb498da0a232b7f04b3 not found: ID does not exist" containerID="e5f220c01510984cfc55b0408efbddadaea1b68eee58ecb498da0a232b7f04b3" Jul 29 14:37:52.530803 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.530762 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5f220c01510984cfc55b0408efbddadaea1b68eee58ecb498da0a232b7f04b3"} err="failed to get container status \"e5f220c01510984cfc55b0408efbddadaea1b68eee58ecb498da0a232b7f04b3\": rpc error: code = NotFound desc = could not find container \"e5f220c01510984cfc55b0408efbddadaea1b68eee58ecb498da0a232b7f04b3\": container with ID starting with e5f220c01510984cfc55b0408efbddadaea1b68eee58ecb498da0a232b7f04b3 not found: ID does not exist" Jul 29 14:37:52.530803 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.530773 2580 scope.go:117] "RemoveContainer" containerID="526e24694b54afbfc20f344231ae1d8419373fd0e878f163fcc02c9cfc636f44" Jul 29 14:37:52.530984 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:37:52.530965 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"526e24694b54afbfc20f344231ae1d8419373fd0e878f163fcc02c9cfc636f44\": container with ID starting with 526e24694b54afbfc20f344231ae1d8419373fd0e878f163fcc02c9cfc636f44 not found: ID does not exist" containerID="526e24694b54afbfc20f344231ae1d8419373fd0e878f163fcc02c9cfc636f44" Jul 29 14:37:52.531025 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.530993 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"526e24694b54afbfc20f344231ae1d8419373fd0e878f163fcc02c9cfc636f44"} err="failed to get container status \"526e24694b54afbfc20f344231ae1d8419373fd0e878f163fcc02c9cfc636f44\": rpc error: code = NotFound desc = could not find container \"526e24694b54afbfc20f344231ae1d8419373fd0e878f163fcc02c9cfc636f44\": container with ID starting with 526e24694b54afbfc20f344231ae1d8419373fd0e878f163fcc02c9cfc636f44 not found: ID does not exist" Jul 29 14:37:52.531060 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.531031 2580 scope.go:117] "RemoveContainer" containerID="1edab304cf606337d47ef4073865b10e1af68b32763f760b451c4c43917ba836" Jul 29 14:37:52.531258 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:37:52.531241 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1edab304cf606337d47ef4073865b10e1af68b32763f760b451c4c43917ba836\": container with ID starting with 1edab304cf606337d47ef4073865b10e1af68b32763f760b451c4c43917ba836 not found: ID does not exist" containerID="1edab304cf606337d47ef4073865b10e1af68b32763f760b451c4c43917ba836" Jul 29 14:37:52.531334 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.531264 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1edab304cf606337d47ef4073865b10e1af68b32763f760b451c4c43917ba836"} err="failed to get container status \"1edab304cf606337d47ef4073865b10e1af68b32763f760b451c4c43917ba836\": rpc error: code = NotFound desc = could not find container \"1edab304cf606337d47ef4073865b10e1af68b32763f760b451c4c43917ba836\": container with ID starting with 1edab304cf606337d47ef4073865b10e1af68b32763f760b451c4c43917ba836 not found: ID does not exist" Jul 29 14:37:52.531334 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.531284 2580 scope.go:117] "RemoveContainer" containerID="0cb48b4df24e2fea384386cf2c1401554bfe1c8c7622e37962fedc8c346c131c" Jul 29 14:37:52.531518 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:37:52.531503 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cb48b4df24e2fea384386cf2c1401554bfe1c8c7622e37962fedc8c346c131c\": container with ID starting with 0cb48b4df24e2fea384386cf2c1401554bfe1c8c7622e37962fedc8c346c131c not found: ID does not exist" containerID="0cb48b4df24e2fea384386cf2c1401554bfe1c8c7622e37962fedc8c346c131c" Jul 29 14:37:52.531559 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.531521 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cb48b4df24e2fea384386cf2c1401554bfe1c8c7622e37962fedc8c346c131c"} err="failed to get container status \"0cb48b4df24e2fea384386cf2c1401554bfe1c8c7622e37962fedc8c346c131c\": rpc error: code = NotFound desc = could not find container \"0cb48b4df24e2fea384386cf2c1401554bfe1c8c7622e37962fedc8c346c131c\": container with ID starting with 0cb48b4df24e2fea384386cf2c1401554bfe1c8c7622e37962fedc8c346c131c not found: ID does not exist" Jul 29 14:37:52.531559 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.531533 2580 scope.go:117] "RemoveContainer" containerID="32a528beb823da80867e358a4bff51b181a2b590c1709087843f5f00cfb40c24" Jul 29 14:37:52.531731 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:37:52.531708 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32a528beb823da80867e358a4bff51b181a2b590c1709087843f5f00cfb40c24\": container with ID starting with 32a528beb823da80867e358a4bff51b181a2b590c1709087843f5f00cfb40c24 not found: ID does not exist" containerID="32a528beb823da80867e358a4bff51b181a2b590c1709087843f5f00cfb40c24" Jul 29 14:37:52.531787 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.531738 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32a528beb823da80867e358a4bff51b181a2b590c1709087843f5f00cfb40c24"} err="failed to get container status \"32a528beb823da80867e358a4bff51b181a2b590c1709087843f5f00cfb40c24\": rpc error: code = NotFound desc = could not find container \"32a528beb823da80867e358a4bff51b181a2b590c1709087843f5f00cfb40c24\": container with ID starting with 32a528beb823da80867e358a4bff51b181a2b590c1709087843f5f00cfb40c24 not found: ID does not exist" Jul 29 14:37:52.531787 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.531760 2580 scope.go:117] "RemoveContainer" containerID="2944a2c30b7bb718f240ee796a05bf159dd86dbb974865705c4fb2157bc005cf" Jul 29 14:37:52.531968 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.531948 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2944a2c30b7bb718f240ee796a05bf159dd86dbb974865705c4fb2157bc005cf"} err="failed to get container status \"2944a2c30b7bb718f240ee796a05bf159dd86dbb974865705c4fb2157bc005cf\": rpc error: code = NotFound desc = could not find container \"2944a2c30b7bb718f240ee796a05bf159dd86dbb974865705c4fb2157bc005cf\": container with ID starting with 2944a2c30b7bb718f240ee796a05bf159dd86dbb974865705c4fb2157bc005cf not found: ID does not exist" Jul 29 14:37:52.532013 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.531977 2580 scope.go:117] "RemoveContainer" containerID="34905cef389b268d13a2d51cb0ea9d976845558ab33bc6c96659384916446203" Jul 29 14:37:52.532147 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.532125 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34905cef389b268d13a2d51cb0ea9d976845558ab33bc6c96659384916446203"} err="failed to get container status \"34905cef389b268d13a2d51cb0ea9d976845558ab33bc6c96659384916446203\": rpc error: code = NotFound desc = could not find container \"34905cef389b268d13a2d51cb0ea9d976845558ab33bc6c96659384916446203\": container with ID starting with 34905cef389b268d13a2d51cb0ea9d976845558ab33bc6c96659384916446203 not found: ID does not exist" Jul 29 14:37:52.532191 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.532147 2580 scope.go:117] "RemoveContainer" containerID="e5f220c01510984cfc55b0408efbddadaea1b68eee58ecb498da0a232b7f04b3" Jul 29 14:37:52.532337 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.532319 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5f220c01510984cfc55b0408efbddadaea1b68eee58ecb498da0a232b7f04b3"} err="failed to get container status \"e5f220c01510984cfc55b0408efbddadaea1b68eee58ecb498da0a232b7f04b3\": rpc error: code = NotFound desc = could not find container \"e5f220c01510984cfc55b0408efbddadaea1b68eee58ecb498da0a232b7f04b3\": container with ID starting with e5f220c01510984cfc55b0408efbddadaea1b68eee58ecb498da0a232b7f04b3 not found: ID does not exist" Jul 29 14:37:52.532373 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.532340 2580 scope.go:117] "RemoveContainer" containerID="526e24694b54afbfc20f344231ae1d8419373fd0e878f163fcc02c9cfc636f44" Jul 29 14:37:52.532567 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.532552 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"526e24694b54afbfc20f344231ae1d8419373fd0e878f163fcc02c9cfc636f44"} err="failed to get container status \"526e24694b54afbfc20f344231ae1d8419373fd0e878f163fcc02c9cfc636f44\": rpc error: code = NotFound desc = could not find container \"526e24694b54afbfc20f344231ae1d8419373fd0e878f163fcc02c9cfc636f44\": container with ID starting with 526e24694b54afbfc20f344231ae1d8419373fd0e878f163fcc02c9cfc636f44 not found: ID does not exist" Jul 29 14:37:52.532607 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.532567 2580 scope.go:117] "RemoveContainer" containerID="1edab304cf606337d47ef4073865b10e1af68b32763f760b451c4c43917ba836" Jul 29 14:37:52.532794 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.532777 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1edab304cf606337d47ef4073865b10e1af68b32763f760b451c4c43917ba836"} err="failed to get container status \"1edab304cf606337d47ef4073865b10e1af68b32763f760b451c4c43917ba836\": rpc error: code = NotFound desc = could not find container \"1edab304cf606337d47ef4073865b10e1af68b32763f760b451c4c43917ba836\": container with ID starting with 1edab304cf606337d47ef4073865b10e1af68b32763f760b451c4c43917ba836 not found: ID does not exist" Jul 29 14:37:52.532794 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.532794 2580 scope.go:117] "RemoveContainer" containerID="0cb48b4df24e2fea384386cf2c1401554bfe1c8c7622e37962fedc8c346c131c" Jul 29 14:37:52.532965 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.532948 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cb48b4df24e2fea384386cf2c1401554bfe1c8c7622e37962fedc8c346c131c"} err="failed to get container status \"0cb48b4df24e2fea384386cf2c1401554bfe1c8c7622e37962fedc8c346c131c\": rpc error: code = NotFound desc = could not find container \"0cb48b4df24e2fea384386cf2c1401554bfe1c8c7622e37962fedc8c346c131c\": container with ID starting with 0cb48b4df24e2fea384386cf2c1401554bfe1c8c7622e37962fedc8c346c131c not found: ID does not exist" Jul 29 14:37:52.533010 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.532965 2580 scope.go:117] "RemoveContainer" containerID="32a528beb823da80867e358a4bff51b181a2b590c1709087843f5f00cfb40c24" Jul 29 14:37:52.533171 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.533156 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32a528beb823da80867e358a4bff51b181a2b590c1709087843f5f00cfb40c24"} err="failed to get container status \"32a528beb823da80867e358a4bff51b181a2b590c1709087843f5f00cfb40c24\": rpc error: code = NotFound desc = could not find container \"32a528beb823da80867e358a4bff51b181a2b590c1709087843f5f00cfb40c24\": container with ID starting with 32a528beb823da80867e358a4bff51b181a2b590c1709087843f5f00cfb40c24 not found: ID does not exist" Jul 29 14:37:52.538649 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.538629 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 29 14:37:52.538895 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.538882 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="kube-rbac-proxy-web" Jul 29 14:37:52.538945 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.538898 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="kube-rbac-proxy-web" Jul 29 14:37:52.538945 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.538905 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="kube-rbac-proxy-metric" Jul 29 14:37:52.538945 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.538911 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="kube-rbac-proxy-metric" Jul 29 14:37:52.538945 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.538919 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="prom-label-proxy" Jul 29 14:37:52.538945 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.538925 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="prom-label-proxy" Jul 29 14:37:52.538945 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.538936 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="init-config-reloader" Jul 29 14:37:52.538945 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.538941 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="init-config-reloader" Jul 29 14:37:52.538945 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.538947 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="config-reloader" Jul 29 14:37:52.539213 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.538952 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="config-reloader" Jul 29 14:37:52.539213 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.538958 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="kube-rbac-proxy" Jul 29 14:37:52.539213 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.538963 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="kube-rbac-proxy" Jul 29 14:37:52.539213 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.538971 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="alertmanager" Jul 29 14:37:52.539213 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.538976 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="alertmanager" Jul 29 14:37:52.539213 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.539013 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="alertmanager" Jul 29 14:37:52.539213 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.539020 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="kube-rbac-proxy-web" Jul 29 14:37:52.539213 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.539025 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="config-reloader" Jul 29 14:37:52.539213 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.539030 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="kube-rbac-proxy" Jul 29 14:37:52.539213 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.539035 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="prom-label-proxy" Jul 29 14:37:52.539213 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.539040 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" containerName="kube-rbac-proxy-metric" Jul 29 14:37:52.541396 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.541382 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.543294 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.543277 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 29 14:37:52.543501 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.543485 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 29 14:37:52.543610 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.543577 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 29 14:37:52.544521 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.544055 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 29 14:37:52.544521 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.544170 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 29 14:37:52.544521 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.544510 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5hvmz\"" Jul 29 14:37:52.544711 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.544545 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 29 14:37:52.544711 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.544685 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 29 14:37:52.546910 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.546592 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 29 14:37:52.551902 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.551881 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 29 14:37:52.557973 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.557954 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 29 14:37:52.702320 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.702266 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af869d30-2596-4b43-a83c-6c8d0e0e332f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.702320 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.702292 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af869d30-2596-4b43-a83c-6c8d0e0e332f-config-out\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.702320 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.702311 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af869d30-2596-4b43-a83c-6c8d0e0e332f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.702461 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.702328 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/af869d30-2596-4b43-a83c-6c8d0e0e332f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.702461 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.702347 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af869d30-2596-4b43-a83c-6c8d0e0e332f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.702461 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.702439 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af869d30-2596-4b43-a83c-6c8d0e0e332f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.702548 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.702463 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af869d30-2596-4b43-a83c-6c8d0e0e332f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.702548 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.702482 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw4d2\" (UniqueName: \"kubernetes.io/projected/af869d30-2596-4b43-a83c-6c8d0e0e332f-kube-api-access-nw4d2\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.702548 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.702516 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/af869d30-2596-4b43-a83c-6c8d0e0e332f-config-volume\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.702548 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.702531 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/af869d30-2596-4b43-a83c-6c8d0e0e332f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.702658 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.702551 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/af869d30-2596-4b43-a83c-6c8d0e0e332f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.702658 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.702583 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af869d30-2596-4b43-a83c-6c8d0e0e332f-web-config\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.702658 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.702602 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/af869d30-2596-4b43-a83c-6c8d0e0e332f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.803390 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.803362 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af869d30-2596-4b43-a83c-6c8d0e0e332f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.803390 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.803390 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af869d30-2596-4b43-a83c-6c8d0e0e332f-config-out\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.803519 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.803408 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af869d30-2596-4b43-a83c-6c8d0e0e332f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.803519 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.803428 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/af869d30-2596-4b43-a83c-6c8d0e0e332f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.803519 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.803448 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af869d30-2596-4b43-a83c-6c8d0e0e332f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.803625 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.803520 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af869d30-2596-4b43-a83c-6c8d0e0e332f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.803625 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.803567 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af869d30-2596-4b43-a83c-6c8d0e0e332f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.803625 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.803601 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw4d2\" (UniqueName: \"kubernetes.io/projected/af869d30-2596-4b43-a83c-6c8d0e0e332f-kube-api-access-nw4d2\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.803801 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.803640 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/af869d30-2596-4b43-a83c-6c8d0e0e332f-config-volume\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.803801 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.803663 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/af869d30-2596-4b43-a83c-6c8d0e0e332f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.803801 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.803692 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/af869d30-2596-4b43-a83c-6c8d0e0e332f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.803801 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.803744 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af869d30-2596-4b43-a83c-6c8d0e0e332f-web-config\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.803801 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.803775 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/af869d30-2596-4b43-a83c-6c8d0e0e332f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.804248 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.804220 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/af869d30-2596-4b43-a83c-6c8d0e0e332f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.804804 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.804775 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af869d30-2596-4b43-a83c-6c8d0e0e332f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.806480 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.806417 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af869d30-2596-4b43-a83c-6c8d0e0e332f-config-out\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.806619 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.806600 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/af869d30-2596-4b43-a83c-6c8d0e0e332f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.807103 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.806756 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af869d30-2596-4b43-a83c-6c8d0e0e332f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.807103 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.807069 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af869d30-2596-4b43-a83c-6c8d0e0e332f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.807251 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.807103 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af869d30-2596-4b43-a83c-6c8d0e0e332f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.807251 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.807071 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/af869d30-2596-4b43-a83c-6c8d0e0e332f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.807251 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.807163 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af869d30-2596-4b43-a83c-6c8d0e0e332f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.807426 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.807397 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/af869d30-2596-4b43-a83c-6c8d0e0e332f-config-volume\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.807556 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.807535 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/af869d30-2596-4b43-a83c-6c8d0e0e332f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.808809 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.808788 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af869d30-2596-4b43-a83c-6c8d0e0e332f-web-config\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.812430 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.812408 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw4d2\" (UniqueName: \"kubernetes.io/projected/af869d30-2596-4b43-a83c-6c8d0e0e332f-kube-api-access-nw4d2\") pod \"alertmanager-main-0\" (UID: \"af869d30-2596-4b43-a83c-6c8d0e0e332f\") " pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.853824 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.853796 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 29 14:37:52.987272 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:52.987198 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 29 14:37:52.989978 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:37:52.989953 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf869d30_2596_4b43_a83c_6c8d0e0e332f.slice/crio-32b09c34d596f378ca09080d1b1e1152d5be8d7b9e10a9c93e58458f987d201b WatchSource:0}: Error finding container 32b09c34d596f378ca09080d1b1e1152d5be8d7b9e10a9c93e58458f987d201b: Status 404 returned error can't find the container with id 32b09c34d596f378ca09080d1b1e1152d5be8d7b9e10a9c93e58458f987d201b Jul 29 14:37:53.491175 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:53.491143 2580 generic.go:356] "Generic (PLEG): container finished" podID="af869d30-2596-4b43-a83c-6c8d0e0e332f" containerID="95737d0edafb85683f41f3f3113939aa09ae502837c3e01130df1a22c3759ceb" exitCode=0 Jul 29 14:37:53.491502 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:53.491241 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af869d30-2596-4b43-a83c-6c8d0e0e332f","Type":"ContainerDied","Data":"95737d0edafb85683f41f3f3113939aa09ae502837c3e01130df1a22c3759ceb"} Jul 29 14:37:53.491502 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:53.491315 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af869d30-2596-4b43-a83c-6c8d0e0e332f","Type":"ContainerStarted","Data":"32b09c34d596f378ca09080d1b1e1152d5be8d7b9e10a9c93e58458f987d201b"} Jul 29 14:37:53.816607 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:53.816581 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48b307f0-dd22-4daf-b297-9b8edd927821" path="/var/lib/kubelet/pods/48b307f0-dd22-4daf-b297-9b8edd927821/volumes" Jul 29 14:37:54.499159 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:54.499124 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af869d30-2596-4b43-a83c-6c8d0e0e332f","Type":"ContainerStarted","Data":"8f3c406efee04f1f8d5d84a2fe5c423d1214b9b6b494c4a0699b512138f4d7a4"} Jul 29 14:37:54.499564 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:54.499234 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af869d30-2596-4b43-a83c-6c8d0e0e332f","Type":"ContainerStarted","Data":"e6aabe2e236966285d19b7026b216a64592459808f5b0e3ddbabdd647537ecca"} Jul 29 14:37:54.499564 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:54.499287 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af869d30-2596-4b43-a83c-6c8d0e0e332f","Type":"ContainerStarted","Data":"560ca6729492605da834220368e95d5e3b263df5f5ef5c16afa5ca488b9afc94"} Jul 29 14:37:54.499564 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:54.499325 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af869d30-2596-4b43-a83c-6c8d0e0e332f","Type":"ContainerStarted","Data":"2c5f33c7277310454b70ec7e87fcc1c6a71d7290c1ecfdcf484e97b450249e97"} Jul 29 14:37:54.499564 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:54.499418 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af869d30-2596-4b43-a83c-6c8d0e0e332f","Type":"ContainerStarted","Data":"52e922970740a08da88891d72ab5c26ab37ffecd012e21808b60a78aac860700"} Jul 29 14:37:54.499564 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:54.499461 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"af869d30-2596-4b43-a83c-6c8d0e0e332f","Type":"ContainerStarted","Data":"0fdf34f17db250581c32c9a3c41e74b52c5ed180264e65b82a1c9f3e6343fb0d"} Jul 29 14:37:54.534843 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:54.534769 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.51787224 podStartE2EDuration="2.534749868s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.877628ms" isStatefulPod=false podCreationTimestamp="2026-07-29 14:37:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:37:54.534525652 +0000 UTC m=+151.262674841" watchObservedRunningTime="2026-07-29 14:37:54.534749868 +0000 UTC m=+151.262899059" Jul 29 14:37:55.100558 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.100525 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn"] Jul 29 14:37:55.102799 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.102778 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.108018 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.107996 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-vq9xs\"" Jul 29 14:37:55.108464 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.108443 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 29 14:37:55.108464 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.108455 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 29 14:37:55.108694 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.108532 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 29 14:37:55.108694 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.108649 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 29 14:37:55.108694 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.108649 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 29 14:37:55.114432 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.114411 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 29 14:37:55.116828 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.116807 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn"] Jul 29 14:37:55.220474 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.220447 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7eac229a-af90-4946-88a6-4c776c96ee51-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.220573 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.220479 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7eac229a-af90-4946-88a6-4c776c96ee51-secret-telemeter-client\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.220573 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.220498 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7eac229a-af90-4946-88a6-4c776c96ee51-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.220573 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.220558 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7eac229a-af90-4946-88a6-4c776c96ee51-telemeter-client-tls\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.220685 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.220579 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7eac229a-af90-4946-88a6-4c776c96ee51-serving-certs-ca-bundle\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.220685 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.220605 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr4gw\" (UniqueName: \"kubernetes.io/projected/7eac229a-af90-4946-88a6-4c776c96ee51-kube-api-access-pr4gw\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.220685 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.220635 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7eac229a-af90-4946-88a6-4c776c96ee51-federate-client-tls\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.220685 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.220652 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eac229a-af90-4946-88a6-4c776c96ee51-metrics-client-ca\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.321317 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.321264 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7eac229a-af90-4946-88a6-4c776c96ee51-telemeter-client-tls\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.321430 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.321321 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7eac229a-af90-4946-88a6-4c776c96ee51-serving-certs-ca-bundle\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.321430 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.321348 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr4gw\" (UniqueName: \"kubernetes.io/projected/7eac229a-af90-4946-88a6-4c776c96ee51-kube-api-access-pr4gw\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.321430 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.321368 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7eac229a-af90-4946-88a6-4c776c96ee51-federate-client-tls\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.321430 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.321393 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eac229a-af90-4946-88a6-4c776c96ee51-metrics-client-ca\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.321430 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.321431 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7eac229a-af90-4946-88a6-4c776c96ee51-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.321693 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.321450 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7eac229a-af90-4946-88a6-4c776c96ee51-secret-telemeter-client\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.321693 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.321476 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7eac229a-af90-4946-88a6-4c776c96ee51-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.322188 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.322159 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7eac229a-af90-4946-88a6-4c776c96ee51-serving-certs-ca-bundle\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.322369 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.322338 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7eac229a-af90-4946-88a6-4c776c96ee51-metrics-client-ca\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.322934 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.322903 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7eac229a-af90-4946-88a6-4c776c96ee51-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.324310 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.324275 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7eac229a-af90-4946-88a6-4c776c96ee51-secret-telemeter-client\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.324671 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.324641 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7eac229a-af90-4946-88a6-4c776c96ee51-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.324788 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.324696 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7eac229a-af90-4946-88a6-4c776c96ee51-federate-client-tls\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.324788 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.324781 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7eac229a-af90-4946-88a6-4c776c96ee51-telemeter-client-tls\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.328782 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.328757 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr4gw\" (UniqueName: \"kubernetes.io/projected/7eac229a-af90-4946-88a6-4c776c96ee51-kube-api-access-pr4gw\") pod \"telemeter-client-6d8d6fc794-xfmrn\" (UID: \"7eac229a-af90-4946-88a6-4c776c96ee51\") " pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.412272 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.412219 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" Jul 29 14:37:55.541823 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:55.541791 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn"] Jul 29 14:37:55.545552 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:37:55.545516 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eac229a_af90_4946_88a6_4c776c96ee51.slice/crio-b16a98be79b7ba91be8d17ee008e307bbdcc9e4a79d8c57f12e37d9518b00d81 WatchSource:0}: Error finding container b16a98be79b7ba91be8d17ee008e307bbdcc9e4a79d8c57f12e37d9518b00d81: Status 404 returned error can't find the container with id b16a98be79b7ba91be8d17ee008e307bbdcc9e4a79d8c57f12e37d9518b00d81 Jul 29 14:37:56.508328 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:56.508290 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" event={"ID":"7eac229a-af90-4946-88a6-4c776c96ee51","Type":"ContainerStarted","Data":"b16a98be79b7ba91be8d17ee008e307bbdcc9e4a79d8c57f12e37d9518b00d81"} Jul 29 14:37:59.522406 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:59.522367 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" event={"ID":"7eac229a-af90-4946-88a6-4c776c96ee51","Type":"ContainerStarted","Data":"ff8390dba46674ed81784a654d79b53c342424a44baae496e79afa2e89b73a1d"} Jul 29 14:37:59.522817 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:59.522440 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" event={"ID":"7eac229a-af90-4946-88a6-4c776c96ee51","Type":"ContainerStarted","Data":"2e7a29af96592daef598aba9237c36532a4eca1f10553acd489473001d7cec2e"} Jul 29 14:37:59.522817 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:59.522470 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" event={"ID":"7eac229a-af90-4946-88a6-4c776c96ee51","Type":"ContainerStarted","Data":"95f084340a9fbcccfd545a3ae327d9a28781007ebd1123defe7bbae4bc73c3ce"} Jul 29 14:37:59.542791 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:37:59.542706 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6d8d6fc794-xfmrn" podStartSLOduration=1.6168791919999999 podStartE2EDuration="4.542670743s" totalImagesPullingTime="2.925791551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:37:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:37:59.541305281 +0000 UTC m=+156.269454472" watchObservedRunningTime="2026-07-29 14:37:59.542670743 +0000 UTC m=+156.270819932" Jul 29 14:38:00.548945 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.548915 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67f5c6b44b-vtwnw"] Jul 29 14:38:00.550965 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.550943 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.562169 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.562139 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67f5c6b44b-vtwnw"] Jul 29 14:38:00.661507 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.661482 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-oauth-serving-cert\") pod \"console-67f5c6b44b-vtwnw\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.661610 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.661538 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-trusted-ca-bundle\") pod \"console-67f5c6b44b-vtwnw\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.661610 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.661585 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d31a0f01-41fc-4937-ade8-70ddde00da97-console-serving-cert\") pod \"console-67f5c6b44b-vtwnw\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.661776 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.661749 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-service-ca\") pod \"console-67f5c6b44b-vtwnw\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.661826 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.661794 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-console-config\") pod \"console-67f5c6b44b-vtwnw\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.661885 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.661821 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xggpk\" (UniqueName: \"kubernetes.io/projected/d31a0f01-41fc-4937-ade8-70ddde00da97-kube-api-access-xggpk\") pod \"console-67f5c6b44b-vtwnw\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.661885 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.661846 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d31a0f01-41fc-4937-ade8-70ddde00da97-console-oauth-config\") pod \"console-67f5c6b44b-vtwnw\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.666003 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.665979 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:38:00.762309 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.762284 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xggpk\" (UniqueName: \"kubernetes.io/projected/d31a0f01-41fc-4937-ade8-70ddde00da97-kube-api-access-xggpk\") pod \"console-67f5c6b44b-vtwnw\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.762309 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.762311 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d31a0f01-41fc-4937-ade8-70ddde00da97-console-oauth-config\") pod \"console-67f5c6b44b-vtwnw\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.762478 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.762341 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-oauth-serving-cert\") pod \"console-67f5c6b44b-vtwnw\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.762478 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.762375 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-trusted-ca-bundle\") pod \"console-67f5c6b44b-vtwnw\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.762478 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.762404 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d31a0f01-41fc-4937-ade8-70ddde00da97-console-serving-cert\") pod \"console-67f5c6b44b-vtwnw\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.762478 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.762453 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-service-ca\") pod \"console-67f5c6b44b-vtwnw\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.762685 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.762482 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-console-config\") pod \"console-67f5c6b44b-vtwnw\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.763292 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.763265 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-console-config\") pod \"console-67f5c6b44b-vtwnw\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.763292 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.763287 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-oauth-serving-cert\") pod \"console-67f5c6b44b-vtwnw\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.763513 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.763484 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-service-ca\") pod \"console-67f5c6b44b-vtwnw\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.763592 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.763566 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-trusted-ca-bundle\") pod \"console-67f5c6b44b-vtwnw\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.765268 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.765245 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d31a0f01-41fc-4937-ade8-70ddde00da97-console-oauth-config\") pod \"console-67f5c6b44b-vtwnw\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.765343 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.765321 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d31a0f01-41fc-4937-ade8-70ddde00da97-console-serving-cert\") pod \"console-67f5c6b44b-vtwnw\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.767146 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.767106 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:38:00.769433 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.769405 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xggpk\" (UniqueName: \"kubernetes.io/projected/d31a0f01-41fc-4937-ade8-70ddde00da97-kube-api-access-xggpk\") pod \"console-67f5c6b44b-vtwnw\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.818960 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.818886 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:38:00.860650 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.860623 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:00.979245 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:00.979225 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67f5c6b44b-vtwnw"] Jul 29 14:38:00.981514 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:38:00.981487 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd31a0f01_41fc_4937_ade8_70ddde00da97.slice/crio-36729e05d88db0cb5e67c7a1ab8c7ce7c89c1e9a89de2fad053c05171b6f2356 WatchSource:0}: Error finding container 36729e05d88db0cb5e67c7a1ab8c7ce7c89c1e9a89de2fad053c05171b6f2356: Status 404 returned error can't find the container with id 36729e05d88db0cb5e67c7a1ab8c7ce7c89c1e9a89de2fad053c05171b6f2356 Jul 29 14:38:01.379451 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:01.379421 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:38:01.430115 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:01.430083 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:38:01.530789 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:01.530752 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67f5c6b44b-vtwnw" event={"ID":"d31a0f01-41fc-4937-ade8-70ddde00da97","Type":"ContainerStarted","Data":"a89624fdcd76cbc6b223e29cb845512aa545f18000957058dad68d0e5977f58b"} Jul 29 14:38:01.530936 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:01.530800 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67f5c6b44b-vtwnw" event={"ID":"d31a0f01-41fc-4937-ade8-70ddde00da97","Type":"ContainerStarted","Data":"36729e05d88db0cb5e67c7a1ab8c7ce7c89c1e9a89de2fad053c05171b6f2356"} Jul 29 14:38:01.548323 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:01.548278 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67f5c6b44b-vtwnw" podStartSLOduration=1.548263366 podStartE2EDuration="1.548263366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:38:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:38:01.546899082 +0000 UTC m=+158.275048263" watchObservedRunningTime="2026-07-29 14:38:01.548263366 +0000 UTC m=+158.276412554" Jul 29 14:38:10.861828 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:10.861791 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:10.862248 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:10.861889 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:10.866430 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:10.866405 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:11.571292 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:11.571263 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:38:11.614825 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:11.614788 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dd5f88445-z44jz"] Jul 29 14:38:11.718151 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:11.718114 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:38:11.819682 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:11.819613 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:38:12.946967 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:12.946936 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:38:35.924995 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:35.924963 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:38:35.985276 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:35.985241 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:38:36.636474 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:36.636433 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-dd5f88445-z44jz" podUID="5007118f-0fad-4036-b794-e942e6d52d1a" containerName="console" containerID="cri-o://58e2e76d0a2ed8e6b6c875845964e162bdfb01b09c8f3ecfff1ecf95f1aa8667" gracePeriod=15 Jul 29 14:38:36.876104 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:36.876080 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:38:36.921468 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:36.921393 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-trusted-ca-bundle\") pod \"5007118f-0fad-4036-b794-e942e6d52d1a\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " Jul 29 14:38:36.921468 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:36.921441 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-service-ca\") pod \"5007118f-0fad-4036-b794-e942e6d52d1a\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " Jul 29 14:38:36.921649 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:36.921477 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtg4z\" (UniqueName: \"kubernetes.io/projected/5007118f-0fad-4036-b794-e942e6d52d1a-kube-api-access-qtg4z\") pod \"5007118f-0fad-4036-b794-e942e6d52d1a\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " Jul 29 14:38:36.921649 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:36.921503 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-console-config\") pod \"5007118f-0fad-4036-b794-e942e6d52d1a\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " Jul 29 14:38:36.921778 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:36.921680 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-oauth-serving-cert\") pod \"5007118f-0fad-4036-b794-e942e6d52d1a\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " Jul 29 14:38:36.921778 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:36.921749 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5007118f-0fad-4036-b794-e942e6d52d1a-console-oauth-config\") pod \"5007118f-0fad-4036-b794-e942e6d52d1a\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " Jul 29 14:38:36.921874 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:36.921812 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5007118f-0fad-4036-b794-e942e6d52d1a-console-serving-cert\") pod \"5007118f-0fad-4036-b794-e942e6d52d1a\" (UID: \"5007118f-0fad-4036-b794-e942e6d52d1a\") " Jul 29 14:38:36.922002 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:36.921972 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-console-config" (OuterVolumeSpecName: "console-config") pod "5007118f-0fad-4036-b794-e942e6d52d1a" (UID: "5007118f-0fad-4036-b794-e942e6d52d1a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 14:38:36.922122 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:36.922004 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5007118f-0fad-4036-b794-e942e6d52d1a" (UID: "5007118f-0fad-4036-b794-e942e6d52d1a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 14:38:36.922122 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:36.922006 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-service-ca" (OuterVolumeSpecName: "service-ca") pod "5007118f-0fad-4036-b794-e942e6d52d1a" (UID: "5007118f-0fad-4036-b794-e942e6d52d1a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 14:38:36.922224 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:36.922126 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5007118f-0fad-4036-b794-e942e6d52d1a" (UID: "5007118f-0fad-4036-b794-e942e6d52d1a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 14:38:36.923915 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:36.923891 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5007118f-0fad-4036-b794-e942e6d52d1a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5007118f-0fad-4036-b794-e942e6d52d1a" (UID: "5007118f-0fad-4036-b794-e942e6d52d1a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 14:38:36.924025 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:36.923935 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5007118f-0fad-4036-b794-e942e6d52d1a-kube-api-access-qtg4z" (OuterVolumeSpecName: "kube-api-access-qtg4z") pod "5007118f-0fad-4036-b794-e942e6d52d1a" (UID: "5007118f-0fad-4036-b794-e942e6d52d1a"). InnerVolumeSpecName "kube-api-access-qtg4z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 14:38:36.924025 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:36.923975 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5007118f-0fad-4036-b794-e942e6d52d1a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5007118f-0fad-4036-b794-e942e6d52d1a" (UID: "5007118f-0fad-4036-b794-e942e6d52d1a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 14:38:37.023370 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:37.023334 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-oauth-serving-cert\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:38:37.023370 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:37.023363 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5007118f-0fad-4036-b794-e942e6d52d1a-console-oauth-config\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:38:37.023370 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:37.023372 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5007118f-0fad-4036-b794-e942e6d52d1a-console-serving-cert\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:38:37.023795 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:37.023381 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-trusted-ca-bundle\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:38:37.023795 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:37.023389 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-service-ca\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:38:37.023795 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:37.023397 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtg4z\" (UniqueName: \"kubernetes.io/projected/5007118f-0fad-4036-b794-e942e6d52d1a-kube-api-access-qtg4z\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:38:37.023795 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:37.023406 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5007118f-0fad-4036-b794-e942e6d52d1a-console-config\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:38:37.660234 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:37.660197 2580 generic.go:356] "Generic (PLEG): container finished" podID="5007118f-0fad-4036-b794-e942e6d52d1a" containerID="58e2e76d0a2ed8e6b6c875845964e162bdfb01b09c8f3ecfff1ecf95f1aa8667" exitCode=0 Jul 29 14:38:37.660395 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:37.660246 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dd5f88445-z44jz" Jul 29 14:38:37.660395 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:37.660259 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dd5f88445-z44jz" event={"ID":"5007118f-0fad-4036-b794-e942e6d52d1a","Type":"ContainerDied","Data":"58e2e76d0a2ed8e6b6c875845964e162bdfb01b09c8f3ecfff1ecf95f1aa8667"} Jul 29 14:38:37.660395 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:37.660307 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dd5f88445-z44jz" event={"ID":"5007118f-0fad-4036-b794-e942e6d52d1a","Type":"ContainerDied","Data":"9b4c8841338d9870d6eaa423cf34933574d5c61d706d3ad45a8ee54ec7a15f89"} Jul 29 14:38:37.660395 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:37.660334 2580 scope.go:117] "RemoveContainer" containerID="58e2e76d0a2ed8e6b6c875845964e162bdfb01b09c8f3ecfff1ecf95f1aa8667" Jul 29 14:38:37.668316 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:37.668296 2580 scope.go:117] "RemoveContainer" containerID="58e2e76d0a2ed8e6b6c875845964e162bdfb01b09c8f3ecfff1ecf95f1aa8667" Jul 29 14:38:37.668559 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:38:37.668540 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58e2e76d0a2ed8e6b6c875845964e162bdfb01b09c8f3ecfff1ecf95f1aa8667\": container with ID starting with 58e2e76d0a2ed8e6b6c875845964e162bdfb01b09c8f3ecfff1ecf95f1aa8667 not found: ID does not exist" containerID="58e2e76d0a2ed8e6b6c875845964e162bdfb01b09c8f3ecfff1ecf95f1aa8667" Jul 29 14:38:37.668599 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:37.668568 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58e2e76d0a2ed8e6b6c875845964e162bdfb01b09c8f3ecfff1ecf95f1aa8667"} err="failed to get container status \"58e2e76d0a2ed8e6b6c875845964e162bdfb01b09c8f3ecfff1ecf95f1aa8667\": rpc error: code = NotFound desc = could not find container \"58e2e76d0a2ed8e6b6c875845964e162bdfb01b09c8f3ecfff1ecf95f1aa8667\": container with ID starting with 58e2e76d0a2ed8e6b6c875845964e162bdfb01b09c8f3ecfff1ecf95f1aa8667 not found: ID does not exist" Jul 29 14:38:37.679242 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:37.679219 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dd5f88445-z44jz"] Jul 29 14:38:37.683914 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:37.683894 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-dd5f88445-z44jz"] Jul 29 14:38:37.817225 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:37.817189 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5007118f-0fad-4036-b794-e942e6d52d1a" path="/var/lib/kubelet/pods/5007118f-0fad-4036-b794-e942e6d52d1a/volumes" Jul 29 14:38:37.989184 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:38:37.989103 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:39:05.269227 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:05.269185 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:39:05.322823 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:05.322786 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:39:15.450193 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.450163 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-69467bccf7-cpbq6"] Jul 29 14:39:15.450631 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.450444 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5007118f-0fad-4036-b794-e942e6d52d1a" containerName="console" Jul 29 14:39:15.450631 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.450455 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5007118f-0fad-4036-b794-e942e6d52d1a" containerName="console" Jul 29 14:39:15.450631 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.450513 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5007118f-0fad-4036-b794-e942e6d52d1a" containerName="console" Jul 29 14:39:15.453266 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.453249 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.465091 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.465067 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69467bccf7-cpbq6"] Jul 29 14:39:15.565864 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.565830 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:39:15.617926 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.617891 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:39:15.634898 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.634876 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-service-ca\") pod \"console-69467bccf7-cpbq6\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.634973 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.634915 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-console-serving-cert\") pod \"console-69467bccf7-cpbq6\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.634973 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.634934 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rwzp\" (UniqueName: \"kubernetes.io/projected/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-kube-api-access-5rwzp\") pod \"console-69467bccf7-cpbq6\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.635041 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.635015 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-console-oauth-config\") pod \"console-69467bccf7-cpbq6\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.635074 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.635046 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-trusted-ca-bundle\") pod \"console-69467bccf7-cpbq6\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.635115 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.635091 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-console-config\") pod \"console-69467bccf7-cpbq6\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.635149 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.635126 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-oauth-serving-cert\") pod \"console-69467bccf7-cpbq6\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.736305 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.736201 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-console-serving-cert\") pod \"console-69467bccf7-cpbq6\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.736305 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.736257 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rwzp\" (UniqueName: \"kubernetes.io/projected/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-kube-api-access-5rwzp\") pod \"console-69467bccf7-cpbq6\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.736305 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.736291 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-console-oauth-config\") pod \"console-69467bccf7-cpbq6\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.736559 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.736320 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-trusted-ca-bundle\") pod \"console-69467bccf7-cpbq6\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.736559 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.736368 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-console-config\") pod \"console-69467bccf7-cpbq6\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.736559 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.736410 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-oauth-serving-cert\") pod \"console-69467bccf7-cpbq6\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.736559 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.736435 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-service-ca\") pod \"console-69467bccf7-cpbq6\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.737188 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.737163 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-service-ca\") pod \"console-69467bccf7-cpbq6\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.737323 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.737183 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-console-config\") pod \"console-69467bccf7-cpbq6\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.737323 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.737247 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-trusted-ca-bundle\") pod \"console-69467bccf7-cpbq6\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.737323 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.737270 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-oauth-serving-cert\") pod \"console-69467bccf7-cpbq6\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.738717 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.738696 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-console-oauth-config\") pod \"console-69467bccf7-cpbq6\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.738717 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.738714 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-console-serving-cert\") pod \"console-69467bccf7-cpbq6\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.743858 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.743840 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rwzp\" (UniqueName: \"kubernetes.io/projected/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-kube-api-access-5rwzp\") pod \"console-69467bccf7-cpbq6\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.761730 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.761705 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:15.846951 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.846914 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:39:15.888600 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.888569 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69467bccf7-cpbq6"] Jul 29 14:39:15.892807 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:39:15.892780 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3b0ea3f_aa66_468d_8fa1_42b1268935c2.slice/crio-8adb8668217c472fb070e76b554edf67076775c5fbd8fb2a7a72a286d70bb39b WatchSource:0}: Error finding container 8adb8668217c472fb070e76b554edf67076775c5fbd8fb2a7a72a286d70bb39b: Status 404 returned error can't find the container with id 8adb8668217c472fb070e76b554edf67076775c5fbd8fb2a7a72a286d70bb39b Jul 29 14:39:15.954025 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:15.953987 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:39:16.473221 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:16.473187 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:39:16.523596 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:16.523564 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:39:16.803300 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:16.803268 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69467bccf7-cpbq6" event={"ID":"d3b0ea3f-aa66-468d-8fa1-42b1268935c2","Type":"ContainerStarted","Data":"1fc1547fbbb5188982ffa30fb30ea6c295be88266c691507290508c12f1bf2cb"} Jul 29 14:39:16.803503 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:16.803323 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69467bccf7-cpbq6" event={"ID":"d3b0ea3f-aa66-468d-8fa1-42b1268935c2","Type":"ContainerStarted","Data":"8adb8668217c472fb070e76b554edf67076775c5fbd8fb2a7a72a286d70bb39b"} Jul 29 14:39:16.826283 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:16.826227 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-69467bccf7-cpbq6" podStartSLOduration=1.826209008 podStartE2EDuration="1.826209008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:39:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:39:16.824547674 +0000 UTC m=+233.552696863" watchObservedRunningTime="2026-07-29 14:39:16.826209008 +0000 UTC m=+233.554358198" Jul 29 14:39:17.192075 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:17.191994 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:39:17.253281 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:17.253250 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:39:25.762064 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:25.762031 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:25.762450 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:25.762088 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:25.766983 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:25.766960 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:25.839177 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:25.839152 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:39:25.884544 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:25.884512 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67f5c6b44b-vtwnw"] Jul 29 14:39:25.989297 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:25.989264 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:39:27.039121 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:27.039077 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:39:27.137993 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:27.137950 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:39:31.619530 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:31.619497 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:39:31.720461 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:31.720428 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:39:50.902997 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:50.902895 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-67f5c6b44b-vtwnw" podUID="d31a0f01-41fc-4937-ade8-70ddde00da97" containerName="console" containerID="cri-o://a89624fdcd76cbc6b223e29cb845512aa545f18000957058dad68d0e5977f58b" gracePeriod=15 Jul 29 14:39:51.137735 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.137702 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:39:51.220470 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.220382 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d31a0f01-41fc-4937-ade8-70ddde00da97-console-oauth-config\") pod \"d31a0f01-41fc-4937-ade8-70ddde00da97\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " Jul 29 14:39:51.220470 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.220432 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-service-ca\") pod \"d31a0f01-41fc-4937-ade8-70ddde00da97\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " Jul 29 14:39:51.220470 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.220469 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xggpk\" (UniqueName: \"kubernetes.io/projected/d31a0f01-41fc-4937-ade8-70ddde00da97-kube-api-access-xggpk\") pod \"d31a0f01-41fc-4937-ade8-70ddde00da97\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " Jul 29 14:39:51.220711 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.220572 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-oauth-serving-cert\") pod \"d31a0f01-41fc-4937-ade8-70ddde00da97\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " Jul 29 14:39:51.220711 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.220624 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-console-config\") pod \"d31a0f01-41fc-4937-ade8-70ddde00da97\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " Jul 29 14:39:51.220711 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.220650 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-trusted-ca-bundle\") pod \"d31a0f01-41fc-4937-ade8-70ddde00da97\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " Jul 29 14:39:51.220711 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.220675 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d31a0f01-41fc-4937-ade8-70ddde00da97-console-serving-cert\") pod \"d31a0f01-41fc-4937-ade8-70ddde00da97\" (UID: \"d31a0f01-41fc-4937-ade8-70ddde00da97\") " Jul 29 14:39:51.221043 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.220996 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-service-ca" (OuterVolumeSpecName: "service-ca") pod "d31a0f01-41fc-4937-ade8-70ddde00da97" (UID: "d31a0f01-41fc-4937-ade8-70ddde00da97"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 14:39:51.221043 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.221021 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-console-config" (OuterVolumeSpecName: "console-config") pod "d31a0f01-41fc-4937-ade8-70ddde00da97" (UID: "d31a0f01-41fc-4937-ade8-70ddde00da97"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 14:39:51.221227 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.221040 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d31a0f01-41fc-4937-ade8-70ddde00da97" (UID: "d31a0f01-41fc-4937-ade8-70ddde00da97"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 14:39:51.221227 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.221066 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d31a0f01-41fc-4937-ade8-70ddde00da97" (UID: "d31a0f01-41fc-4937-ade8-70ddde00da97"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 14:39:51.222623 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.222598 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d31a0f01-41fc-4937-ade8-70ddde00da97-kube-api-access-xggpk" (OuterVolumeSpecName: "kube-api-access-xggpk") pod "d31a0f01-41fc-4937-ade8-70ddde00da97" (UID: "d31a0f01-41fc-4937-ade8-70ddde00da97"). InnerVolumeSpecName "kube-api-access-xggpk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 14:39:51.222745 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.222597 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d31a0f01-41fc-4937-ade8-70ddde00da97-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d31a0f01-41fc-4937-ade8-70ddde00da97" (UID: "d31a0f01-41fc-4937-ade8-70ddde00da97"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 14:39:51.222745 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.222654 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d31a0f01-41fc-4937-ade8-70ddde00da97-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d31a0f01-41fc-4937-ade8-70ddde00da97" (UID: "d31a0f01-41fc-4937-ade8-70ddde00da97"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 14:39:51.322226 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.322184 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xggpk\" (UniqueName: \"kubernetes.io/projected/d31a0f01-41fc-4937-ade8-70ddde00da97-kube-api-access-xggpk\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:39:51.322226 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.322220 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-oauth-serving-cert\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:39:51.322226 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.322230 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-console-config\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:39:51.322457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.322239 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-trusted-ca-bundle\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:39:51.322457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.322249 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d31a0f01-41fc-4937-ade8-70ddde00da97-console-serving-cert\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:39:51.322457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.322257 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d31a0f01-41fc-4937-ade8-70ddde00da97-console-oauth-config\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:39:51.322457 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.322265 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d31a0f01-41fc-4937-ade8-70ddde00da97-service-ca\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:39:51.928094 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.928059 2580 generic.go:356] "Generic (PLEG): container finished" podID="d31a0f01-41fc-4937-ade8-70ddde00da97" containerID="a89624fdcd76cbc6b223e29cb845512aa545f18000957058dad68d0e5977f58b" exitCode=0 Jul 29 14:39:51.928534 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.928105 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67f5c6b44b-vtwnw" Jul 29 14:39:51.928534 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.928149 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67f5c6b44b-vtwnw" event={"ID":"d31a0f01-41fc-4937-ade8-70ddde00da97","Type":"ContainerDied","Data":"a89624fdcd76cbc6b223e29cb845512aa545f18000957058dad68d0e5977f58b"} Jul 29 14:39:51.928534 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.928201 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67f5c6b44b-vtwnw" event={"ID":"d31a0f01-41fc-4937-ade8-70ddde00da97","Type":"ContainerDied","Data":"36729e05d88db0cb5e67c7a1ab8c7ce7c89c1e9a89de2fad053c05171b6f2356"} Jul 29 14:39:51.928534 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.928222 2580 scope.go:117] "RemoveContainer" containerID="a89624fdcd76cbc6b223e29cb845512aa545f18000957058dad68d0e5977f58b" Jul 29 14:39:51.935949 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.935932 2580 scope.go:117] "RemoveContainer" containerID="a89624fdcd76cbc6b223e29cb845512aa545f18000957058dad68d0e5977f58b" Jul 29 14:39:51.936212 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:39:51.936191 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a89624fdcd76cbc6b223e29cb845512aa545f18000957058dad68d0e5977f58b\": container with ID starting with a89624fdcd76cbc6b223e29cb845512aa545f18000957058dad68d0e5977f58b not found: ID does not exist" containerID="a89624fdcd76cbc6b223e29cb845512aa545f18000957058dad68d0e5977f58b" Jul 29 14:39:51.936258 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.936220 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a89624fdcd76cbc6b223e29cb845512aa545f18000957058dad68d0e5977f58b"} err="failed to get container status \"a89624fdcd76cbc6b223e29cb845512aa545f18000957058dad68d0e5977f58b\": rpc error: code = NotFound desc = could not find container \"a89624fdcd76cbc6b223e29cb845512aa545f18000957058dad68d0e5977f58b\": container with ID starting with a89624fdcd76cbc6b223e29cb845512aa545f18000957058dad68d0e5977f58b not found: ID does not exist" Jul 29 14:39:51.942839 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.942807 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67f5c6b44b-vtwnw"] Jul 29 14:39:51.949079 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:51.949053 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67f5c6b44b-vtwnw"] Jul 29 14:39:53.816666 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:39:53.816631 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d31a0f01-41fc-4937-ade8-70ddde00da97" path="/var/lib/kubelet/pods/d31a0f01-41fc-4937-ade8-70ddde00da97/volumes" Jul 29 14:40:19.061448 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:19.061410 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:40:19.794452 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:19.794420 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw"] Jul 29 14:40:19.794747 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:19.794734 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d31a0f01-41fc-4937-ade8-70ddde00da97" containerName="console" Jul 29 14:40:19.794797 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:19.794748 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d31a0f01-41fc-4937-ade8-70ddde00da97" containerName="console" Jul 29 14:40:19.794828 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:19.794800 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d31a0f01-41fc-4937-ade8-70ddde00da97" containerName="console" Jul 29 14:40:19.799415 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:19.799375 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw" Jul 29 14:40:19.801564 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:19.801542 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 29 14:40:19.801682 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:19.801549 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 29 14:40:19.801927 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:19.801911 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v5w7z\"" Jul 29 14:40:19.806973 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:19.806950 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw"] Jul 29 14:40:19.848626 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:19.848593 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhtxg\" (UniqueName: \"kubernetes.io/projected/0d4b19f4-a9be-4ccc-9db7-f41ef2897226-kube-api-access-xhtxg\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw\" (UID: \"0d4b19f4-a9be-4ccc-9db7-f41ef2897226\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw" Jul 29 14:40:19.848811 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:19.848649 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d4b19f4-a9be-4ccc-9db7-f41ef2897226-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw\" (UID: \"0d4b19f4-a9be-4ccc-9db7-f41ef2897226\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw" Jul 29 14:40:19.848811 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:19.848698 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d4b19f4-a9be-4ccc-9db7-f41ef2897226-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw\" (UID: \"0d4b19f4-a9be-4ccc-9db7-f41ef2897226\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw" Jul 29 14:40:19.949099 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:19.949063 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d4b19f4-a9be-4ccc-9db7-f41ef2897226-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw\" (UID: \"0d4b19f4-a9be-4ccc-9db7-f41ef2897226\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw" Jul 29 14:40:19.949268 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:19.949152 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhtxg\" (UniqueName: \"kubernetes.io/projected/0d4b19f4-a9be-4ccc-9db7-f41ef2897226-kube-api-access-xhtxg\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw\" (UID: \"0d4b19f4-a9be-4ccc-9db7-f41ef2897226\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw" Jul 29 14:40:19.949268 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:19.949193 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d4b19f4-a9be-4ccc-9db7-f41ef2897226-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw\" (UID: \"0d4b19f4-a9be-4ccc-9db7-f41ef2897226\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw" Jul 29 14:40:19.949462 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:19.949441 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d4b19f4-a9be-4ccc-9db7-f41ef2897226-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw\" (UID: \"0d4b19f4-a9be-4ccc-9db7-f41ef2897226\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw" Jul 29 14:40:19.949535 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:19.949515 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d4b19f4-a9be-4ccc-9db7-f41ef2897226-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw\" (UID: \"0d4b19f4-a9be-4ccc-9db7-f41ef2897226\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw" Jul 29 14:40:19.956962 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:19.956938 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhtxg\" (UniqueName: \"kubernetes.io/projected/0d4b19f4-a9be-4ccc-9db7-f41ef2897226-kube-api-access-xhtxg\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw\" (UID: \"0d4b19f4-a9be-4ccc-9db7-f41ef2897226\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw" Jul 29 14:40:20.109866 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:20.109770 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw" Jul 29 14:40:20.229451 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:20.229419 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw"] Jul 29 14:40:20.232285 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:40:20.232256 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d4b19f4_a9be_4ccc_9db7_f41ef2897226.slice/crio-80b392756a700be67f19c1dcd3fd270cd761a20c82e0ddb479e2cc2977659a5a WatchSource:0}: Error finding container 80b392756a700be67f19c1dcd3fd270cd761a20c82e0ddb479e2cc2977659a5a: Status 404 returned error can't find the container with id 80b392756a700be67f19c1dcd3fd270cd761a20c82e0ddb479e2cc2977659a5a Jul 29 14:40:21.031789 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:21.031750 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw" event={"ID":"0d4b19f4-a9be-4ccc-9db7-f41ef2897226","Type":"ContainerStarted","Data":"80b392756a700be67f19c1dcd3fd270cd761a20c82e0ddb479e2cc2977659a5a"} Jul 29 14:40:21.796543 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:21.796502 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:40:23.666001 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:23.665971 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 14:40:23.671946 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:23.671921 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 29 14:40:23.672870 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:23.672849 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 14:40:27.057447 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:27.057415 2580 generic.go:356] "Generic (PLEG): container finished" podID="0d4b19f4-a9be-4ccc-9db7-f41ef2897226" containerID="881622a292f3f1cb7e1f3355a304fab7675d757b3a936523dbefdf4b75c4f063" exitCode=0 Jul 29 14:40:27.057817 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:27.057479 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw" event={"ID":"0d4b19f4-a9be-4ccc-9db7-f41ef2897226","Type":"ContainerDied","Data":"881622a292f3f1cb7e1f3355a304fab7675d757b3a936523dbefdf4b75c4f063"} Jul 29 14:40:27.058436 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:27.058415 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 14:40:30.069335 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:30.069300 2580 generic.go:356] "Generic (PLEG): container finished" podID="0d4b19f4-a9be-4ccc-9db7-f41ef2897226" containerID="9c02a0c1e4113e3ad4677be58d2b623afa63a4a2e156b437956f70b7ece945cd" exitCode=0 Jul 29 14:40:30.069761 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:30.069377 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw" event={"ID":"0d4b19f4-a9be-4ccc-9db7-f41ef2897226","Type":"ContainerDied","Data":"9c02a0c1e4113e3ad4677be58d2b623afa63a4a2e156b437956f70b7ece945cd"} Jul 29 14:40:37.099143 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:37.099101 2580 generic.go:356] "Generic (PLEG): container finished" podID="0d4b19f4-a9be-4ccc-9db7-f41ef2897226" containerID="9232a9795da9a53ca533c6931de633c1b7e782e85d971041d10829f34a7f9f0b" exitCode=0 Jul 29 14:40:37.099509 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:37.099168 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw" event={"ID":"0d4b19f4-a9be-4ccc-9db7-f41ef2897226","Type":"ContainerDied","Data":"9232a9795da9a53ca533c6931de633c1b7e782e85d971041d10829f34a7f9f0b"} Jul 29 14:40:38.225661 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:38.225637 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw" Jul 29 14:40:38.292303 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:38.292267 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhtxg\" (UniqueName: \"kubernetes.io/projected/0d4b19f4-a9be-4ccc-9db7-f41ef2897226-kube-api-access-xhtxg\") pod \"0d4b19f4-a9be-4ccc-9db7-f41ef2897226\" (UID: \"0d4b19f4-a9be-4ccc-9db7-f41ef2897226\") " Jul 29 14:40:38.292467 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:38.292322 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d4b19f4-a9be-4ccc-9db7-f41ef2897226-util\") pod \"0d4b19f4-a9be-4ccc-9db7-f41ef2897226\" (UID: \"0d4b19f4-a9be-4ccc-9db7-f41ef2897226\") " Jul 29 14:40:38.292467 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:38.292363 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d4b19f4-a9be-4ccc-9db7-f41ef2897226-bundle\") pod \"0d4b19f4-a9be-4ccc-9db7-f41ef2897226\" (UID: \"0d4b19f4-a9be-4ccc-9db7-f41ef2897226\") " Jul 29 14:40:38.293002 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:38.292981 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d4b19f4-a9be-4ccc-9db7-f41ef2897226-bundle" (OuterVolumeSpecName: "bundle") pod "0d4b19f4-a9be-4ccc-9db7-f41ef2897226" (UID: "0d4b19f4-a9be-4ccc-9db7-f41ef2897226"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:40:38.294458 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:38.294432 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d4b19f4-a9be-4ccc-9db7-f41ef2897226-kube-api-access-xhtxg" (OuterVolumeSpecName: "kube-api-access-xhtxg") pod "0d4b19f4-a9be-4ccc-9db7-f41ef2897226" (UID: "0d4b19f4-a9be-4ccc-9db7-f41ef2897226"). InnerVolumeSpecName "kube-api-access-xhtxg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 14:40:38.296508 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:38.296483 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d4b19f4-a9be-4ccc-9db7-f41ef2897226-util" (OuterVolumeSpecName: "util") pod "0d4b19f4-a9be-4ccc-9db7-f41ef2897226" (UID: "0d4b19f4-a9be-4ccc-9db7-f41ef2897226"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:40:38.393933 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:38.393833 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d4b19f4-a9be-4ccc-9db7-f41ef2897226-util\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:40:38.393933 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:38.393873 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d4b19f4-a9be-4ccc-9db7-f41ef2897226-bundle\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:40:38.393933 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:38.393883 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhtxg\" (UniqueName: \"kubernetes.io/projected/0d4b19f4-a9be-4ccc-9db7-f41ef2897226-kube-api-access-xhtxg\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:40:39.108170 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:39.108134 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw" event={"ID":"0d4b19f4-a9be-4ccc-9db7-f41ef2897226","Type":"ContainerDied","Data":"80b392756a700be67f19c1dcd3fd270cd761a20c82e0ddb479e2cc2977659a5a"} Jul 29 14:40:39.108170 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:39.108156 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vfctw" Jul 29 14:40:39.108374 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:39.108187 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="80b392756a700be67f19c1dcd3fd270cd761a20c82e0ddb479e2cc2977659a5a" Jul 29 14:40:40.783124 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:40.783069 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:40:41.644010 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:41.643961 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:40:42.766349 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:42.766309 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2l682"] Jul 29 14:40:42.766927 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:42.766708 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d4b19f4-a9be-4ccc-9db7-f41ef2897226" containerName="extract" Jul 29 14:40:42.766927 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:42.766741 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d4b19f4-a9be-4ccc-9db7-f41ef2897226" containerName="extract" Jul 29 14:40:42.766927 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:42.766773 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d4b19f4-a9be-4ccc-9db7-f41ef2897226" containerName="util" Jul 29 14:40:42.766927 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:42.766782 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d4b19f4-a9be-4ccc-9db7-f41ef2897226" containerName="util" Jul 29 14:40:42.766927 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:42.766793 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d4b19f4-a9be-4ccc-9db7-f41ef2897226" containerName="pull" Jul 29 14:40:42.766927 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:42.766801 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d4b19f4-a9be-4ccc-9db7-f41ef2897226" containerName="pull" Jul 29 14:40:42.766927 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:42.766874 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d4b19f4-a9be-4ccc-9db7-f41ef2897226" containerName="extract" Jul 29 14:40:42.769425 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:42.769404 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2l682" Jul 29 14:40:42.771768 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:42.771746 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 29 14:40:42.771961 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:42.771949 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 29 14:40:42.772275 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:42.772259 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-zw2nh\"" Jul 29 14:40:42.788920 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:42.788898 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2l682"] Jul 29 14:40:42.832153 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:42.832124 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5msr\" (UniqueName: \"kubernetes.io/projected/f8b4e12c-ba06-461a-8294-91746578a69b-kube-api-access-c5msr\") pod \"cert-manager-operator-controller-manager-55c444bf86-2l682\" (UID: \"f8b4e12c-ba06-461a-8294-91746578a69b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2l682" Jul 29 14:40:42.832321 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:42.832169 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8b4e12c-ba06-461a-8294-91746578a69b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2l682\" (UID: \"f8b4e12c-ba06-461a-8294-91746578a69b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2l682" Jul 29 14:40:42.932718 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:42.932671 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5msr\" (UniqueName: \"kubernetes.io/projected/f8b4e12c-ba06-461a-8294-91746578a69b-kube-api-access-c5msr\") pod \"cert-manager-operator-controller-manager-55c444bf86-2l682\" (UID: \"f8b4e12c-ba06-461a-8294-91746578a69b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2l682" Jul 29 14:40:42.932906 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:42.932757 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8b4e12c-ba06-461a-8294-91746578a69b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2l682\" (UID: \"f8b4e12c-ba06-461a-8294-91746578a69b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2l682" Jul 29 14:40:42.933111 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:42.933096 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8b4e12c-ba06-461a-8294-91746578a69b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2l682\" (UID: \"f8b4e12c-ba06-461a-8294-91746578a69b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2l682" Jul 29 14:40:42.940389 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:42.940354 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5msr\" (UniqueName: \"kubernetes.io/projected/f8b4e12c-ba06-461a-8294-91746578a69b-kube-api-access-c5msr\") pod \"cert-manager-operator-controller-manager-55c444bf86-2l682\" (UID: \"f8b4e12c-ba06-461a-8294-91746578a69b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2l682" Jul 29 14:40:43.077962 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:43.077867 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2l682" Jul 29 14:40:43.208960 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:43.208926 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2l682"] Jul 29 14:40:43.212430 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:40:43.212390 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8b4e12c_ba06_461a_8294_91746578a69b.slice/crio-1e7ee1d44cda093b669e95e1a52049a0641354c9606d31efde047f3c9b47a71e WatchSource:0}: Error finding container 1e7ee1d44cda093b669e95e1a52049a0641354c9606d31efde047f3c9b47a71e: Status 404 returned error can't find the container with id 1e7ee1d44cda093b669e95e1a52049a0641354c9606d31efde047f3c9b47a71e Jul 29 14:40:44.129253 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:44.129214 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2l682" event={"ID":"f8b4e12c-ba06-461a-8294-91746578a69b","Type":"ContainerStarted","Data":"1e7ee1d44cda093b669e95e1a52049a0641354c9606d31efde047f3c9b47a71e"} Jul 29 14:40:44.398478 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:44.398389 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:40:45.135509 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:45.135474 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2l682" event={"ID":"f8b4e12c-ba06-461a-8294-91746578a69b","Type":"ContainerStarted","Data":"a5745fe0d44a1a480bea1d271680f94b7071c7650ec7ce3c9a5aa7ecb3cec051"} Jul 29 14:40:45.156964 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:45.156910 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2l682" podStartSLOduration=1.380500168 podStartE2EDuration="3.15688428s" totalImagesPullingTime="1.776384112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:40:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:40:45.155818029 +0000 UTC m=+321.883967221" watchObservedRunningTime="2026-07-29 14:40:45.15688428 +0000 UTC m=+321.885033469" Jul 29 14:40:45.169466 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:45.169427 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:40:45.578690 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:45.578634 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:40:45.684940 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:45.684903 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:40:46.414675 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.414642 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22"] Jul 29 14:40:46.417930 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.417913 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22" Jul 29 14:40:46.420509 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.420489 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 29 14:40:46.420609 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.420531 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v5w7z\"" Jul 29 14:40:46.421164 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.421143 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 29 14:40:46.429445 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.429423 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22"] Jul 29 14:40:46.463019 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.462989 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jvmw\" (UniqueName: \"kubernetes.io/projected/8412d2b2-43c9-43d8-ad68-921c86f2e74c-kube-api-access-5jvmw\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22\" (UID: \"8412d2b2-43c9-43d8-ad68-921c86f2e74c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22" Jul 29 14:40:46.463180 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.463043 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8412d2b2-43c9-43d8-ad68-921c86f2e74c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22\" (UID: \"8412d2b2-43c9-43d8-ad68-921c86f2e74c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22" Jul 29 14:40:46.463180 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.463066 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8412d2b2-43c9-43d8-ad68-921c86f2e74c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22\" (UID: \"8412d2b2-43c9-43d8-ad68-921c86f2e74c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22" Jul 29 14:40:46.564130 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.564090 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8412d2b2-43c9-43d8-ad68-921c86f2e74c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22\" (UID: \"8412d2b2-43c9-43d8-ad68-921c86f2e74c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22" Jul 29 14:40:46.564337 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.564145 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8412d2b2-43c9-43d8-ad68-921c86f2e74c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22\" (UID: \"8412d2b2-43c9-43d8-ad68-921c86f2e74c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22" Jul 29 14:40:46.564337 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.564205 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jvmw\" (UniqueName: \"kubernetes.io/projected/8412d2b2-43c9-43d8-ad68-921c86f2e74c-kube-api-access-5jvmw\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22\" (UID: \"8412d2b2-43c9-43d8-ad68-921c86f2e74c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22" Jul 29 14:40:46.564522 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.564503 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8412d2b2-43c9-43d8-ad68-921c86f2e74c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22\" (UID: \"8412d2b2-43c9-43d8-ad68-921c86f2e74c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22" Jul 29 14:40:46.564556 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.564541 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8412d2b2-43c9-43d8-ad68-921c86f2e74c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22\" (UID: \"8412d2b2-43c9-43d8-ad68-921c86f2e74c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22" Jul 29 14:40:46.581234 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.581206 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jvmw\" (UniqueName: \"kubernetes.io/projected/8412d2b2-43c9-43d8-ad68-921c86f2e74c-kube-api-access-5jvmw\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22\" (UID: \"8412d2b2-43c9-43d8-ad68-921c86f2e74c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22" Jul 29 14:40:46.614414 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.614380 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:40:46.721779 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.721681 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:40:46.726669 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.726649 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22" Jul 29 14:40:46.728795 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.728772 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-r7mdj"] Jul 29 14:40:46.731634 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.731619 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-r7mdj" Jul 29 14:40:46.733907 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.733889 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 29 14:40:46.734229 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.734214 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 29 14:40:46.742060 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.742034 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-r7mdj"] Jul 29 14:40:46.856928 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.856895 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22"] Jul 29 14:40:46.860208 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:40:46.860182 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8412d2b2_43c9_43d8_ad68_921c86f2e74c.slice/crio-f34cfead86cb26d045bed34158e93577e044eb4776a9a4d64d11e5e3d2bc6ead WatchSource:0}: Error finding container f34cfead86cb26d045bed34158e93577e044eb4776a9a4d64d11e5e3d2bc6ead: Status 404 returned error can't find the container with id f34cfead86cb26d045bed34158e93577e044eb4776a9a4d64d11e5e3d2bc6ead Jul 29 14:40:46.865767 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.865745 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2e6aadfb-58e7-490a-866d-5a47dc692c47-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-r7mdj\" (UID: \"2e6aadfb-58e7-490a-866d-5a47dc692c47\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r7mdj" Jul 29 14:40:46.865837 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.865777 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ftqm\" (UniqueName: \"kubernetes.io/projected/2e6aadfb-58e7-490a-866d-5a47dc692c47-kube-api-access-8ftqm\") pod \"cert-manager-webhook-6486ffd997-r7mdj\" (UID: \"2e6aadfb-58e7-490a-866d-5a47dc692c47\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r7mdj" Jul 29 14:40:46.967092 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.967052 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2e6aadfb-58e7-490a-866d-5a47dc692c47-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-r7mdj\" (UID: \"2e6aadfb-58e7-490a-866d-5a47dc692c47\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r7mdj" Jul 29 14:40:46.967243 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.967092 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ftqm\" (UniqueName: \"kubernetes.io/projected/2e6aadfb-58e7-490a-866d-5a47dc692c47-kube-api-access-8ftqm\") pod \"cert-manager-webhook-6486ffd997-r7mdj\" (UID: \"2e6aadfb-58e7-490a-866d-5a47dc692c47\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r7mdj" Jul 29 14:40:46.978140 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.978080 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2e6aadfb-58e7-490a-866d-5a47dc692c47-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-r7mdj\" (UID: \"2e6aadfb-58e7-490a-866d-5a47dc692c47\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r7mdj" Jul 29 14:40:46.978320 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:46.978196 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ftqm\" (UniqueName: \"kubernetes.io/projected/2e6aadfb-58e7-490a-866d-5a47dc692c47-kube-api-access-8ftqm\") pod \"cert-manager-webhook-6486ffd997-r7mdj\" (UID: \"2e6aadfb-58e7-490a-866d-5a47dc692c47\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r7mdj" Jul 29 14:40:47.066769 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:47.066717 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-r7mdj" Jul 29 14:40:47.146088 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:47.146057 2580 generic.go:356] "Generic (PLEG): container finished" podID="8412d2b2-43c9-43d8-ad68-921c86f2e74c" containerID="1d3d582c2d4b3ff6d132a859bbfb281105796bfd397e26ee2af7551398d1dc94" exitCode=0 Jul 29 14:40:47.146254 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:47.146145 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22" event={"ID":"8412d2b2-43c9-43d8-ad68-921c86f2e74c","Type":"ContainerDied","Data":"1d3d582c2d4b3ff6d132a859bbfb281105796bfd397e26ee2af7551398d1dc94"} Jul 29 14:40:47.146254 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:47.146194 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22" event={"ID":"8412d2b2-43c9-43d8-ad68-921c86f2e74c","Type":"ContainerStarted","Data":"f34cfead86cb26d045bed34158e93577e044eb4776a9a4d64d11e5e3d2bc6ead"} Jul 29 14:40:47.193777 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:47.193744 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-r7mdj"] Jul 29 14:40:47.196496 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:40:47.196469 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e6aadfb_58e7_490a_866d_5a47dc692c47.slice/crio-dcd6834dbbef8475e4f1464e350eaee866171e43e51368ec7414e97f37d77d94 WatchSource:0}: Error finding container dcd6834dbbef8475e4f1464e350eaee866171e43e51368ec7414e97f37d77d94: Status 404 returned error can't find the container with id dcd6834dbbef8475e4f1464e350eaee866171e43e51368ec7414e97f37d77d94 Jul 29 14:40:48.153298 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:48.153258 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-r7mdj" event={"ID":"2e6aadfb-58e7-490a-866d-5a47dc692c47","Type":"ContainerStarted","Data":"dcd6834dbbef8475e4f1464e350eaee866171e43e51368ec7414e97f37d77d94"} Jul 29 14:40:48.608336 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:48.608221 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:40:49.012827 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:49.012788 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:40:50.164287 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:50.164251 2580 generic.go:356] "Generic (PLEG): container finished" podID="8412d2b2-43c9-43d8-ad68-921c86f2e74c" containerID="96d3e7009960c245678a27521fe0e21c0803e3be6792a6800e6276f85de11f3a" exitCode=0 Jul 29 14:40:50.164674 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:50.164308 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22" event={"ID":"8412d2b2-43c9-43d8-ad68-921c86f2e74c","Type":"ContainerDied","Data":"96d3e7009960c245678a27521fe0e21c0803e3be6792a6800e6276f85de11f3a"} Jul 29 14:40:51.169819 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:51.169779 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-r7mdj" event={"ID":"2e6aadfb-58e7-490a-866d-5a47dc692c47","Type":"ContainerStarted","Data":"566d1e2a3399f3300317d098c71635973543334d20faf5ed2f07fd91162d44f5"} Jul 29 14:40:51.170269 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:51.169908 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-r7mdj" Jul 29 14:40:51.171870 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:51.171849 2580 generic.go:356] "Generic (PLEG): container finished" podID="8412d2b2-43c9-43d8-ad68-921c86f2e74c" containerID="42e876bb4e2a866c58fc54f55200f70c2e0d3e3f664d0c938d0fd34c803869b6" exitCode=0 Jul 29 14:40:51.171986 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:51.171923 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22" event={"ID":"8412d2b2-43c9-43d8-ad68-921c86f2e74c","Type":"ContainerDied","Data":"42e876bb4e2a866c58fc54f55200f70c2e0d3e3f664d0c938d0fd34c803869b6"} Jul 29 14:40:51.189816 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:51.189767 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-r7mdj" podStartSLOduration=1.820976599 podStartE2EDuration="5.18975361s" totalImagesPullingTime="3.368777011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:40:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:40:51.188416877 +0000 UTC m=+327.916566066" watchObservedRunningTime="2026-07-29 14:40:51.18975361 +0000 UTC m=+327.917902797" Jul 29 14:40:51.198206 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:51.198179 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:40:51.260738 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:51.260681 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:40:52.295131 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:52.295108 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22" Jul 29 14:40:52.413329 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:52.413292 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jvmw\" (UniqueName: \"kubernetes.io/projected/8412d2b2-43c9-43d8-ad68-921c86f2e74c-kube-api-access-5jvmw\") pod \"8412d2b2-43c9-43d8-ad68-921c86f2e74c\" (UID: \"8412d2b2-43c9-43d8-ad68-921c86f2e74c\") " Jul 29 14:40:52.413504 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:52.413398 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8412d2b2-43c9-43d8-ad68-921c86f2e74c-bundle\") pod \"8412d2b2-43c9-43d8-ad68-921c86f2e74c\" (UID: \"8412d2b2-43c9-43d8-ad68-921c86f2e74c\") " Jul 29 14:40:52.413504 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:52.413460 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8412d2b2-43c9-43d8-ad68-921c86f2e74c-util\") pod \"8412d2b2-43c9-43d8-ad68-921c86f2e74c\" (UID: \"8412d2b2-43c9-43d8-ad68-921c86f2e74c\") " Jul 29 14:40:52.413809 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:52.413786 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8412d2b2-43c9-43d8-ad68-921c86f2e74c-bundle" (OuterVolumeSpecName: "bundle") pod "8412d2b2-43c9-43d8-ad68-921c86f2e74c" (UID: "8412d2b2-43c9-43d8-ad68-921c86f2e74c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:40:52.415384 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:52.415358 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8412d2b2-43c9-43d8-ad68-921c86f2e74c-kube-api-access-5jvmw" (OuterVolumeSpecName: "kube-api-access-5jvmw") pod "8412d2b2-43c9-43d8-ad68-921c86f2e74c" (UID: "8412d2b2-43c9-43d8-ad68-921c86f2e74c"). InnerVolumeSpecName "kube-api-access-5jvmw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 14:40:52.417247 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:52.417219 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8412d2b2-43c9-43d8-ad68-921c86f2e74c-util" (OuterVolumeSpecName: "util") pod "8412d2b2-43c9-43d8-ad68-921c86f2e74c" (UID: "8412d2b2-43c9-43d8-ad68-921c86f2e74c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:40:52.514342 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:52.514310 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8412d2b2-43c9-43d8-ad68-921c86f2e74c-bundle\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:40:52.514342 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:52.514340 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8412d2b2-43c9-43d8-ad68-921c86f2e74c-util\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:40:52.514528 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:52.514350 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jvmw\" (UniqueName: \"kubernetes.io/projected/8412d2b2-43c9-43d8-ad68-921c86f2e74c-kube-api-access-5jvmw\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:40:53.181465 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:53.181438 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22" Jul 29 14:40:53.181656 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:53.181433 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6g22" event={"ID":"8412d2b2-43c9-43d8-ad68-921c86f2e74c","Type":"ContainerDied","Data":"f34cfead86cb26d045bed34158e93577e044eb4776a9a4d64d11e5e3d2bc6ead"} Jul 29 14:40:53.181656 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:53.181560 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f34cfead86cb26d045bed34158e93577e044eb4776a9a4d64d11e5e3d2bc6ead" Jul 29 14:40:54.771738 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:54.771691 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:40:54.870255 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:54.870204 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:40:56.838205 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:56.838164 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-prjd2"] Jul 29 14:40:56.838609 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:56.838432 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8412d2b2-43c9-43d8-ad68-921c86f2e74c" containerName="pull" Jul 29 14:40:56.838609 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:56.838441 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8412d2b2-43c9-43d8-ad68-921c86f2e74c" containerName="pull" Jul 29 14:40:56.838609 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:56.838456 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8412d2b2-43c9-43d8-ad68-921c86f2e74c" containerName="util" Jul 29 14:40:56.838609 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:56.838462 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8412d2b2-43c9-43d8-ad68-921c86f2e74c" containerName="util" Jul 29 14:40:56.838609 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:56.838475 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8412d2b2-43c9-43d8-ad68-921c86f2e74c" containerName="extract" Jul 29 14:40:56.838609 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:56.838482 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8412d2b2-43c9-43d8-ad68-921c86f2e74c" containerName="extract" Jul 29 14:40:56.838609 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:56.838529 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8412d2b2-43c9-43d8-ad68-921c86f2e74c" containerName="extract" Jul 29 14:40:56.841779 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:56.841756 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-prjd2" Jul 29 14:40:56.843810 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:56.843790 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 29 14:40:56.843929 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:56.843792 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 29 14:40:56.844194 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:56.844175 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-rw4jx\"" Jul 29 14:40:56.850626 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:56.850606 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-prjd2"] Jul 29 14:40:56.947676 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:56.947639 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsq7c\" (UniqueName: \"kubernetes.io/projected/b6cc3a03-14f5-44d1-9595-8f5942242b37-kube-api-access-rsq7c\") pod \"openshift-lws-operator-fd8ccff4c-prjd2\" (UID: \"b6cc3a03-14f5-44d1-9595-8f5942242b37\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-prjd2" Jul 29 14:40:56.947676 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:56.947681 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6cc3a03-14f5-44d1-9595-8f5942242b37-tmp\") pod \"openshift-lws-operator-fd8ccff4c-prjd2\" (UID: \"b6cc3a03-14f5-44d1-9595-8f5942242b37\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-prjd2" Jul 29 14:40:57.048882 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:57.048855 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsq7c\" (UniqueName: \"kubernetes.io/projected/b6cc3a03-14f5-44d1-9595-8f5942242b37-kube-api-access-rsq7c\") pod \"openshift-lws-operator-fd8ccff4c-prjd2\" (UID: \"b6cc3a03-14f5-44d1-9595-8f5942242b37\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-prjd2" Jul 29 14:40:57.049003 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:57.048886 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6cc3a03-14f5-44d1-9595-8f5942242b37-tmp\") pod \"openshift-lws-operator-fd8ccff4c-prjd2\" (UID: \"b6cc3a03-14f5-44d1-9595-8f5942242b37\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-prjd2" Jul 29 14:40:57.049202 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:57.049186 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6cc3a03-14f5-44d1-9595-8f5942242b37-tmp\") pod \"openshift-lws-operator-fd8ccff4c-prjd2\" (UID: \"b6cc3a03-14f5-44d1-9595-8f5942242b37\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-prjd2" Jul 29 14:40:57.057057 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:57.057033 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsq7c\" (UniqueName: \"kubernetes.io/projected/b6cc3a03-14f5-44d1-9595-8f5942242b37-kube-api-access-rsq7c\") pod \"openshift-lws-operator-fd8ccff4c-prjd2\" (UID: \"b6cc3a03-14f5-44d1-9595-8f5942242b37\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-prjd2" Jul 29 14:40:57.151087 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:57.151019 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-prjd2" Jul 29 14:40:57.178539 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:57.178514 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-r7mdj" Jul 29 14:40:57.217199 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:57.217159 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:40:57.284117 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:57.284088 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-prjd2"] Jul 29 14:40:57.286942 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:40:57.286920 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6cc3a03_14f5_44d1_9595_8f5942242b37.slice/crio-f3033ea1ac2e3c3f8c00529ab883b159bab31d7dee956f1eab0e86a35f9ec9e1 WatchSource:0}: Error finding container f3033ea1ac2e3c3f8c00529ab883b159bab31d7dee956f1eab0e86a35f9ec9e1: Status 404 returned error can't find the container with id f3033ea1ac2e3c3f8c00529ab883b159bab31d7dee956f1eab0e86a35f9ec9e1 Jul 29 14:40:57.291960 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:57.291933 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8vqt6"] Jul 29 14:40:57.298002 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:57.297985 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-8vqt6" Jul 29 14:40:57.299883 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:57.299867 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-dtldh\"" Jul 29 14:40:57.305925 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:57.305908 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8vqt6"] Jul 29 14:40:57.453945 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:57.453851 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/11f72d5e-e559-4a8f-ab9a-c082c9e85225-bound-sa-token\") pod \"cert-manager-55b4d954d5-8vqt6\" (UID: \"11f72d5e-e559-4a8f-ab9a-c082c9e85225\") " pod="cert-manager/cert-manager-55b4d954d5-8vqt6" Jul 29 14:40:57.454099 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:57.453970 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xstzc\" (UniqueName: \"kubernetes.io/projected/11f72d5e-e559-4a8f-ab9a-c082c9e85225-kube-api-access-xstzc\") pod \"cert-manager-55b4d954d5-8vqt6\" (UID: \"11f72d5e-e559-4a8f-ab9a-c082c9e85225\") " pod="cert-manager/cert-manager-55b4d954d5-8vqt6" Jul 29 14:40:57.554413 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:57.554383 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/11f72d5e-e559-4a8f-ab9a-c082c9e85225-bound-sa-token\") pod \"cert-manager-55b4d954d5-8vqt6\" (UID: \"11f72d5e-e559-4a8f-ab9a-c082c9e85225\") " pod="cert-manager/cert-manager-55b4d954d5-8vqt6" Jul 29 14:40:57.554567 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:57.554441 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xstzc\" (UniqueName: \"kubernetes.io/projected/11f72d5e-e559-4a8f-ab9a-c082c9e85225-kube-api-access-xstzc\") pod \"cert-manager-55b4d954d5-8vqt6\" (UID: \"11f72d5e-e559-4a8f-ab9a-c082c9e85225\") " pod="cert-manager/cert-manager-55b4d954d5-8vqt6" Jul 29 14:40:57.562311 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:57.562285 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/11f72d5e-e559-4a8f-ab9a-c082c9e85225-bound-sa-token\") pod \"cert-manager-55b4d954d5-8vqt6\" (UID: \"11f72d5e-e559-4a8f-ab9a-c082c9e85225\") " pod="cert-manager/cert-manager-55b4d954d5-8vqt6" Jul 29 14:40:57.562402 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:57.562394 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xstzc\" (UniqueName: \"kubernetes.io/projected/11f72d5e-e559-4a8f-ab9a-c082c9e85225-kube-api-access-xstzc\") pod \"cert-manager-55b4d954d5-8vqt6\" (UID: \"11f72d5e-e559-4a8f-ab9a-c082c9e85225\") " pod="cert-manager/cert-manager-55b4d954d5-8vqt6" Jul 29 14:40:57.608458 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:57.608427 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-8vqt6" Jul 29 14:40:57.737305 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:57.737281 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8vqt6"] Jul 29 14:40:57.739532 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:40:57.739504 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod11f72d5e_e559_4a8f_ab9a_c082c9e85225.slice/crio-e93a8f4811b89702a2f735cb401055f4bee1c20d233db9e87669597fc3776c3d WatchSource:0}: Error finding container e93a8f4811b89702a2f735cb401055f4bee1c20d233db9e87669597fc3776c3d: Status 404 returned error can't find the container with id e93a8f4811b89702a2f735cb401055f4bee1c20d233db9e87669597fc3776c3d Jul 29 14:40:58.204752 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:58.204698 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-8vqt6" event={"ID":"11f72d5e-e559-4a8f-ab9a-c082c9e85225","Type":"ContainerStarted","Data":"893bc9e5c3f3c587920462380bebee34aacb5ca56c786470022c132be7ac1b90"} Jul 29 14:40:58.205174 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:58.204763 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-8vqt6" event={"ID":"11f72d5e-e559-4a8f-ab9a-c082c9e85225","Type":"ContainerStarted","Data":"e93a8f4811b89702a2f735cb401055f4bee1c20d233db9e87669597fc3776c3d"} Jul 29 14:40:58.206537 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:58.206505 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-prjd2" event={"ID":"b6cc3a03-14f5-44d1-9595-8f5942242b37","Type":"ContainerStarted","Data":"f3033ea1ac2e3c3f8c00529ab883b159bab31d7dee956f1eab0e86a35f9ec9e1"} Jul 29 14:40:58.223208 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:58.223152 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-8vqt6" podStartSLOduration=1.223132255 podStartE2EDuration="1.223132255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:40:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:40:58.221183575 +0000 UTC m=+334.949332764" watchObservedRunningTime="2026-07-29 14:40:58.223132255 +0000 UTC m=+334.951281448" Jul 29 14:40:59.213101 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:59.213062 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-prjd2" event={"ID":"b6cc3a03-14f5-44d1-9595-8f5942242b37","Type":"ContainerStarted","Data":"bc08b6da8782bc118e257c35944454d760c267862d2a9658255edd9ba8592b01"} Jul 29 14:40:59.229550 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:59.229502 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-prjd2" podStartSLOduration=1.494027566 podStartE2EDuration="3.229488249s" totalImagesPullingTime="1.735460683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:40:59.227592339 +0000 UTC m=+335.955741525" watchObservedRunningTime="2026-07-29 14:40:59.229488249 +0000 UTC m=+335.957637437" Jul 29 14:40:59.875252 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:59.875222 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:40:59.976611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:40:59.976578 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:01.879264 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:01.879235 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb"] Jul 29 14:41:01.882744 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:01.882711 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb" Jul 29 14:41:01.884691 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:01.884666 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-v5w7z\"" Jul 29 14:41:01.884816 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:01.884688 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 29 14:41:01.884816 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:01.884688 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 29 14:41:01.891351 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:01.891331 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb"] Jul 29 14:41:01.992953 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:01.992917 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/815725d4-30e0-467b-9dd7-16f86e497b49-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb\" (UID: \"815725d4-30e0-467b-9dd7-16f86e497b49\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb" Jul 29 14:41:01.993109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:01.992979 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4mf4\" (UniqueName: \"kubernetes.io/projected/815725d4-30e0-467b-9dd7-16f86e497b49-kube-api-access-s4mf4\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb\" (UID: \"815725d4-30e0-467b-9dd7-16f86e497b49\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb" Jul 29 14:41:01.993109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:01.993062 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/815725d4-30e0-467b-9dd7-16f86e497b49-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb\" (UID: \"815725d4-30e0-467b-9dd7-16f86e497b49\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb" Jul 29 14:41:02.094363 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:02.094330 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/815725d4-30e0-467b-9dd7-16f86e497b49-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb\" (UID: \"815725d4-30e0-467b-9dd7-16f86e497b49\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb" Jul 29 14:41:02.094487 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:02.094384 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/815725d4-30e0-467b-9dd7-16f86e497b49-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb\" (UID: \"815725d4-30e0-467b-9dd7-16f86e497b49\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb" Jul 29 14:41:02.094487 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:02.094413 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4mf4\" (UniqueName: \"kubernetes.io/projected/815725d4-30e0-467b-9dd7-16f86e497b49-kube-api-access-s4mf4\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb\" (UID: \"815725d4-30e0-467b-9dd7-16f86e497b49\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb" Jul 29 14:41:02.094695 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:02.094677 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/815725d4-30e0-467b-9dd7-16f86e497b49-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb\" (UID: \"815725d4-30e0-467b-9dd7-16f86e497b49\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb" Jul 29 14:41:02.094773 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:02.094758 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/815725d4-30e0-467b-9dd7-16f86e497b49-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb\" (UID: \"815725d4-30e0-467b-9dd7-16f86e497b49\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb" Jul 29 14:41:02.102112 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:02.102080 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4mf4\" (UniqueName: \"kubernetes.io/projected/815725d4-30e0-467b-9dd7-16f86e497b49-kube-api-access-s4mf4\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb\" (UID: \"815725d4-30e0-467b-9dd7-16f86e497b49\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb" Jul 29 14:41:02.192298 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:02.192219 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb" Jul 29 14:41:02.202270 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:02.202222 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:02.313896 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:02.313864 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb"] Jul 29 14:41:02.316470 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:41:02.316439 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod815725d4_30e0_467b_9dd7_16f86e497b49.slice/crio-aa3df38a25eb206c8f6b1fe191f7888549ff96c9333bff641eb16d52c8a1dc11 WatchSource:0}: Error finding container aa3df38a25eb206c8f6b1fe191f7888549ff96c9333bff641eb16d52c8a1dc11: Status 404 returned error can't find the container with id aa3df38a25eb206c8f6b1fe191f7888549ff96c9333bff641eb16d52c8a1dc11 Jul 29 14:41:03.232351 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:03.232311 2580 generic.go:356] "Generic (PLEG): container finished" podID="815725d4-30e0-467b-9dd7-16f86e497b49" containerID="d60126e859c58eea186f83e48e38419fb55442af56439ab5c31c65369e9d9574" exitCode=0 Jul 29 14:41:03.232749 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:03.232368 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb" event={"ID":"815725d4-30e0-467b-9dd7-16f86e497b49","Type":"ContainerDied","Data":"d60126e859c58eea186f83e48e38419fb55442af56439ab5c31c65369e9d9574"} Jul 29 14:41:03.232749 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:03.232411 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb" event={"ID":"815725d4-30e0-467b-9dd7-16f86e497b49","Type":"ContainerStarted","Data":"aa3df38a25eb206c8f6b1fe191f7888549ff96c9333bff641eb16d52c8a1dc11"} Jul 29 14:41:05.242566 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:05.242530 2580 generic.go:356] "Generic (PLEG): container finished" podID="815725d4-30e0-467b-9dd7-16f86e497b49" containerID="380d880f6201f3d6a7371ff9b467e25eac64236ee91ec000e671916f0c07bc35" exitCode=0 Jul 29 14:41:05.243010 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:05.242610 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb" event={"ID":"815725d4-30e0-467b-9dd7-16f86e497b49","Type":"ContainerDied","Data":"380d880f6201f3d6a7371ff9b467e25eac64236ee91ec000e671916f0c07bc35"} Jul 29 14:41:06.251098 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:06.251062 2580 generic.go:356] "Generic (PLEG): container finished" podID="815725d4-30e0-467b-9dd7-16f86e497b49" containerID="54885c5db724f751df9265878a9a7bedadac81e9e62235b1134dc830f215eed8" exitCode=0 Jul 29 14:41:06.251582 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:06.251118 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb" event={"ID":"815725d4-30e0-467b-9dd7-16f86e497b49","Type":"ContainerDied","Data":"54885c5db724f751df9265878a9a7bedadac81e9e62235b1134dc830f215eed8"} Jul 29 14:41:07.388955 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:07.388927 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb" Jul 29 14:41:07.539093 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:07.539001 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4mf4\" (UniqueName: \"kubernetes.io/projected/815725d4-30e0-467b-9dd7-16f86e497b49-kube-api-access-s4mf4\") pod \"815725d4-30e0-467b-9dd7-16f86e497b49\" (UID: \"815725d4-30e0-467b-9dd7-16f86e497b49\") " Jul 29 14:41:07.539093 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:07.539051 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/815725d4-30e0-467b-9dd7-16f86e497b49-util\") pod \"815725d4-30e0-467b-9dd7-16f86e497b49\" (UID: \"815725d4-30e0-467b-9dd7-16f86e497b49\") " Jul 29 14:41:07.539275 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:07.539193 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/815725d4-30e0-467b-9dd7-16f86e497b49-bundle\") pod \"815725d4-30e0-467b-9dd7-16f86e497b49\" (UID: \"815725d4-30e0-467b-9dd7-16f86e497b49\") " Jul 29 14:41:07.539904 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:07.539871 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/815725d4-30e0-467b-9dd7-16f86e497b49-bundle" (OuterVolumeSpecName: "bundle") pod "815725d4-30e0-467b-9dd7-16f86e497b49" (UID: "815725d4-30e0-467b-9dd7-16f86e497b49"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:41:07.541131 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:07.541106 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/815725d4-30e0-467b-9dd7-16f86e497b49-kube-api-access-s4mf4" (OuterVolumeSpecName: "kube-api-access-s4mf4") pod "815725d4-30e0-467b-9dd7-16f86e497b49" (UID: "815725d4-30e0-467b-9dd7-16f86e497b49"). InnerVolumeSpecName "kube-api-access-s4mf4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 14:41:07.544398 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:07.544375 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/815725d4-30e0-467b-9dd7-16f86e497b49-util" (OuterVolumeSpecName: "util") pod "815725d4-30e0-467b-9dd7-16f86e497b49" (UID: "815725d4-30e0-467b-9dd7-16f86e497b49"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:41:07.639880 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:07.639846 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4mf4\" (UniqueName: \"kubernetes.io/projected/815725d4-30e0-467b-9dd7-16f86e497b49-kube-api-access-s4mf4\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:41:07.639880 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:07.639875 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/815725d4-30e0-467b-9dd7-16f86e497b49-util\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:41:07.639880 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:07.639885 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/815725d4-30e0-467b-9dd7-16f86e497b49-bundle\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:41:08.260864 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:08.260833 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb" event={"ID":"815725d4-30e0-467b-9dd7-16f86e497b49","Type":"ContainerDied","Data":"aa3df38a25eb206c8f6b1fe191f7888549ff96c9333bff641eb16d52c8a1dc11"} Jul 29 14:41:08.261034 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:08.260880 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afc5kqb" Jul 29 14:41:08.261034 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:08.260886 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa3df38a25eb206c8f6b1fe191f7888549ff96c9333bff641eb16d52c8a1dc11" Jul 29 14:41:21.019208 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:21.019162 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:21.112666 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:21.112616 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:21.610425 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:21.610391 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:21.920651 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:21.920557 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:22.022869 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.022826 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:22.033315 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.033287 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l"] Jul 29 14:41:22.033578 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.033566 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="815725d4-30e0-467b-9dd7-16f86e497b49" containerName="util" Jul 29 14:41:22.033620 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.033579 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="815725d4-30e0-467b-9dd7-16f86e497b49" containerName="util" Jul 29 14:41:22.033620 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.033594 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="815725d4-30e0-467b-9dd7-16f86e497b49" containerName="pull" Jul 29 14:41:22.033620 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.033599 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="815725d4-30e0-467b-9dd7-16f86e497b49" containerName="pull" Jul 29 14:41:22.033620 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.033611 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="815725d4-30e0-467b-9dd7-16f86e497b49" containerName="extract" Jul 29 14:41:22.033620 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.033616 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="815725d4-30e0-467b-9dd7-16f86e497b49" containerName="extract" Jul 29 14:41:22.033794 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.033664 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="815725d4-30e0-467b-9dd7-16f86e497b49" containerName="extract" Jul 29 14:41:22.035422 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.035408 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" Jul 29 14:41:22.039127 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.039103 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 29 14:41:22.039238 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.039110 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 29 14:41:22.040692 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.040672 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 29 14:41:22.040851 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.040834 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 29 14:41:22.040929 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.040913 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zq7td\"" Jul 29 14:41:22.053655 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.053628 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l"] Jul 29 14:41:22.058920 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.058896 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2f5da262-cc15-4b81-b450-11badbd13afd-webhook-cert\") pod \"opendatahub-operator-controller-manager-65f9f6cd8c-78t5l\" (UID: \"2f5da262-cc15-4b81-b450-11badbd13afd\") " pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" Jul 29 14:41:22.059009 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.058943 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f5da262-cc15-4b81-b450-11badbd13afd-tmp\") pod \"opendatahub-operator-controller-manager-65f9f6cd8c-78t5l\" (UID: \"2f5da262-cc15-4b81-b450-11badbd13afd\") " pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" Jul 29 14:41:22.059054 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.059013 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2f5da262-cc15-4b81-b450-11badbd13afd-manifests\") pod \"opendatahub-operator-controller-manager-65f9f6cd8c-78t5l\" (UID: \"2f5da262-cc15-4b81-b450-11badbd13afd\") " pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" Jul 29 14:41:22.059054 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.059038 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2f5da262-cc15-4b81-b450-11badbd13afd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65f9f6cd8c-78t5l\" (UID: \"2f5da262-cc15-4b81-b450-11badbd13afd\") " pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" Jul 29 14:41:22.059125 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.059059 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpf48\" (UniqueName: \"kubernetes.io/projected/2f5da262-cc15-4b81-b450-11badbd13afd-kube-api-access-mpf48\") pod \"opendatahub-operator-controller-manager-65f9f6cd8c-78t5l\" (UID: \"2f5da262-cc15-4b81-b450-11badbd13afd\") " pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" Jul 29 14:41:22.159920 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.159884 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f5da262-cc15-4b81-b450-11badbd13afd-tmp\") pod \"opendatahub-operator-controller-manager-65f9f6cd8c-78t5l\" (UID: \"2f5da262-cc15-4b81-b450-11badbd13afd\") " pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" Jul 29 14:41:22.160105 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.159941 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2f5da262-cc15-4b81-b450-11badbd13afd-manifests\") pod \"opendatahub-operator-controller-manager-65f9f6cd8c-78t5l\" (UID: \"2f5da262-cc15-4b81-b450-11badbd13afd\") " pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" Jul 29 14:41:22.160105 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.159962 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2f5da262-cc15-4b81-b450-11badbd13afd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65f9f6cd8c-78t5l\" (UID: \"2f5da262-cc15-4b81-b450-11badbd13afd\") " pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" Jul 29 14:41:22.160105 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.159985 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpf48\" (UniqueName: \"kubernetes.io/projected/2f5da262-cc15-4b81-b450-11badbd13afd-kube-api-access-mpf48\") pod \"opendatahub-operator-controller-manager-65f9f6cd8c-78t5l\" (UID: \"2f5da262-cc15-4b81-b450-11badbd13afd\") " pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" Jul 29 14:41:22.160105 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.160054 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2f5da262-cc15-4b81-b450-11badbd13afd-webhook-cert\") pod \"opendatahub-operator-controller-manager-65f9f6cd8c-78t5l\" (UID: \"2f5da262-cc15-4b81-b450-11badbd13afd\") " pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" Jul 29 14:41:22.160397 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.160366 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2f5da262-cc15-4b81-b450-11badbd13afd-manifests\") pod \"opendatahub-operator-controller-manager-65f9f6cd8c-78t5l\" (UID: \"2f5da262-cc15-4b81-b450-11badbd13afd\") " pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" Jul 29 14:41:22.162309 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.162289 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f5da262-cc15-4b81-b450-11badbd13afd-tmp\") pod \"opendatahub-operator-controller-manager-65f9f6cd8c-78t5l\" (UID: \"2f5da262-cc15-4b81-b450-11badbd13afd\") " pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" Jul 29 14:41:22.162397 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.162361 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2f5da262-cc15-4b81-b450-11badbd13afd-webhook-cert\") pod \"opendatahub-operator-controller-manager-65f9f6cd8c-78t5l\" (UID: \"2f5da262-cc15-4b81-b450-11badbd13afd\") " pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" Jul 29 14:41:22.162548 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.162531 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2f5da262-cc15-4b81-b450-11badbd13afd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65f9f6cd8c-78t5l\" (UID: \"2f5da262-cc15-4b81-b450-11badbd13afd\") " pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" Jul 29 14:41:22.175776 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.174698 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpf48\" (UniqueName: \"kubernetes.io/projected/2f5da262-cc15-4b81-b450-11badbd13afd-kube-api-access-mpf48\") pod \"opendatahub-operator-controller-manager-65f9f6cd8c-78t5l\" (UID: \"2f5da262-cc15-4b81-b450-11badbd13afd\") " pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" Jul 29 14:41:22.258552 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.258514 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:22.345842 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.345813 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" Jul 29 14:41:22.363109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.363075 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:22.463812 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.463710 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:22.476022 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.475994 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l"] Jul 29 14:41:22.479168 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:41:22.479143 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f5da262_cc15_4b81_b450_11badbd13afd.slice/crio-b2179ae7329900ce56f4babf1981dc6a130c287114b48cecb0cca674ffce5c19 WatchSource:0}: Error finding container b2179ae7329900ce56f4babf1981dc6a130c287114b48cecb0cca674ffce5c19: Status 404 returned error can't find the container with id b2179ae7329900ce56f4babf1981dc6a130c287114b48cecb0cca674ffce5c19 Jul 29 14:41:22.564867 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.564829 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:22.669102 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:22.669064 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:23.330436 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:23.330390 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" event={"ID":"2f5da262-cc15-4b81-b450-11badbd13afd","Type":"ContainerStarted","Data":"b2179ae7329900ce56f4babf1981dc6a130c287114b48cecb0cca674ffce5c19"} Jul 29 14:41:25.340879 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:25.340850 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" event={"ID":"2f5da262-cc15-4b81-b450-11badbd13afd","Type":"ContainerStarted","Data":"ef3c61d0de00a41e635d17fc50914519fd1ed83529f31b6995a8f240eb26f732"} Jul 29 14:41:27.351774 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:27.351732 2580 generic.go:356] "Generic (PLEG): container finished" podID="2f5da262-cc15-4b81-b450-11badbd13afd" containerID="ef3c61d0de00a41e635d17fc50914519fd1ed83529f31b6995a8f240eb26f732" exitCode=0 Jul 29 14:41:27.351774 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:27.351755 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" event={"ID":"2f5da262-cc15-4b81-b450-11badbd13afd","Type":"ContainerDied","Data":"ef3c61d0de00a41e635d17fc50914519fd1ed83529f31b6995a8f240eb26f732"} Jul 29 14:41:28.358010 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:28.357973 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" event={"ID":"2f5da262-cc15-4b81-b450-11badbd13afd","Type":"ContainerStarted","Data":"c5e9bfd9a73e2a68720dd6d432f4ee237e806b354953c776493a2e896eb595c0"} Jul 29 14:41:28.358378 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:28.358203 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" Jul 29 14:41:28.384505 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:28.384443 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" podStartSLOduration=2.22859713 podStartE2EDuration="6.384427651s" totalImagesPullingTime="3.134305198s" totalInitContainerRuntime="1.021525323s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:41:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:41:28.382695873 +0000 UTC m=+365.110845074" watchObservedRunningTime="2026-07-29 14:41:28.384427651 +0000 UTC m=+365.112576840" Jul 29 14:41:31.947252 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:31.947215 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:32.071035 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:32.070997 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:39.364745 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:39.364693 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-65f9f6cd8c-78t5l" Jul 29 14:41:41.093075 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:41.093042 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:41.265286 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:41.265254 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:41.755105 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:41.755071 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:41.789737 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:41.789697 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-68ffcd8f6-t4pfh"] Jul 29 14:41:41.798183 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:41.798159 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68ffcd8f6-t4pfh" Jul 29 14:41:41.800852 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:41.800828 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-gzx2x\"" Jul 29 14:41:41.800982 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:41.800828 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 29 14:41:41.800982 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:41.800829 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 29 14:41:41.801086 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:41.800828 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 29 14:41:41.801227 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:41.801212 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 29 14:41:41.806613 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:41.806592 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68ffcd8f6-t4pfh"] Jul 29 14:41:41.859260 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:41.859225 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:41.912160 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:41.912124 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11d59862-3e7f-4c04-9ca7-fe8cacecadf5-tmp\") pod \"kube-auth-proxy-68ffcd8f6-t4pfh\" (UID: \"11d59862-3e7f-4c04-9ca7-fe8cacecadf5\") " pod="openshift-ingress/kube-auth-proxy-68ffcd8f6-t4pfh" Jul 29 14:41:41.912326 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:41.912174 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99xbb\" (UniqueName: \"kubernetes.io/projected/11d59862-3e7f-4c04-9ca7-fe8cacecadf5-kube-api-access-99xbb\") pod \"kube-auth-proxy-68ffcd8f6-t4pfh\" (UID: \"11d59862-3e7f-4c04-9ca7-fe8cacecadf5\") " pod="openshift-ingress/kube-auth-proxy-68ffcd8f6-t4pfh" Jul 29 14:41:41.912326 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:41.912202 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11d59862-3e7f-4c04-9ca7-fe8cacecadf5-tls-certs\") pod \"kube-auth-proxy-68ffcd8f6-t4pfh\" (UID: \"11d59862-3e7f-4c04-9ca7-fe8cacecadf5\") " pod="openshift-ingress/kube-auth-proxy-68ffcd8f6-t4pfh" Jul 29 14:41:42.013778 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.013668 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11d59862-3e7f-4c04-9ca7-fe8cacecadf5-tmp\") pod \"kube-auth-proxy-68ffcd8f6-t4pfh\" (UID: \"11d59862-3e7f-4c04-9ca7-fe8cacecadf5\") " pod="openshift-ingress/kube-auth-proxy-68ffcd8f6-t4pfh" Jul 29 14:41:42.013778 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.013717 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99xbb\" (UniqueName: \"kubernetes.io/projected/11d59862-3e7f-4c04-9ca7-fe8cacecadf5-kube-api-access-99xbb\") pod \"kube-auth-proxy-68ffcd8f6-t4pfh\" (UID: \"11d59862-3e7f-4c04-9ca7-fe8cacecadf5\") " pod="openshift-ingress/kube-auth-proxy-68ffcd8f6-t4pfh" Jul 29 14:41:42.014000 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.013808 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11d59862-3e7f-4c04-9ca7-fe8cacecadf5-tls-certs\") pod \"kube-auth-proxy-68ffcd8f6-t4pfh\" (UID: \"11d59862-3e7f-4c04-9ca7-fe8cacecadf5\") " pod="openshift-ingress/kube-auth-proxy-68ffcd8f6-t4pfh" Jul 29 14:41:42.015940 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.015911 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11d59862-3e7f-4c04-9ca7-fe8cacecadf5-tmp\") pod \"kube-auth-proxy-68ffcd8f6-t4pfh\" (UID: \"11d59862-3e7f-4c04-9ca7-fe8cacecadf5\") " pod="openshift-ingress/kube-auth-proxy-68ffcd8f6-t4pfh" Jul 29 14:41:42.016145 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.016128 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11d59862-3e7f-4c04-9ca7-fe8cacecadf5-tls-certs\") pod \"kube-auth-proxy-68ffcd8f6-t4pfh\" (UID: \"11d59862-3e7f-4c04-9ca7-fe8cacecadf5\") " pod="openshift-ingress/kube-auth-proxy-68ffcd8f6-t4pfh" Jul 29 14:41:42.022262 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.022236 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mxwh6"] Jul 29 14:41:42.026064 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.026042 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mxwh6" Jul 29 14:41:42.029262 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.029244 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 29 14:41:42.029464 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.029453 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-qq9lk\"" Jul 29 14:41:42.032109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.032089 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99xbb\" (UniqueName: \"kubernetes.io/projected/11d59862-3e7f-4c04-9ca7-fe8cacecadf5-kube-api-access-99xbb\") pod \"kube-auth-proxy-68ffcd8f6-t4pfh\" (UID: \"11d59862-3e7f-4c04-9ca7-fe8cacecadf5\") " pod="openshift-ingress/kube-auth-proxy-68ffcd8f6-t4pfh" Jul 29 14:41:42.044364 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.044323 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mxwh6"] Jul 29 14:41:42.075102 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.075070 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:42.107741 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.107705 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68ffcd8f6-t4pfh" Jul 29 14:41:42.114960 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.114938 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq2m5\" (UniqueName: \"kubernetes.io/projected/637fd95f-e250-422b-9132-580af39a1611-kube-api-access-qq2m5\") pod \"odh-model-controller-7998c67dc4-mxwh6\" (UID: \"637fd95f-e250-422b-9132-580af39a1611\") " pod="opendatahub/odh-model-controller-7998c67dc4-mxwh6" Jul 29 14:41:42.115063 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.114972 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/637fd95f-e250-422b-9132-580af39a1611-cert\") pod \"odh-model-controller-7998c67dc4-mxwh6\" (UID: \"637fd95f-e250-422b-9132-580af39a1611\") " pod="opendatahub/odh-model-controller-7998c67dc4-mxwh6" Jul 29 14:41:42.178288 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.178255 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:42.216306 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.216275 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq2m5\" (UniqueName: \"kubernetes.io/projected/637fd95f-e250-422b-9132-580af39a1611-kube-api-access-qq2m5\") pod \"odh-model-controller-7998c67dc4-mxwh6\" (UID: \"637fd95f-e250-422b-9132-580af39a1611\") " pod="opendatahub/odh-model-controller-7998c67dc4-mxwh6" Jul 29 14:41:42.216454 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.216336 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/637fd95f-e250-422b-9132-580af39a1611-cert\") pod \"odh-model-controller-7998c67dc4-mxwh6\" (UID: \"637fd95f-e250-422b-9132-580af39a1611\") " pod="opendatahub/odh-model-controller-7998c67dc4-mxwh6" Jul 29 14:41:42.216528 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:41:42.216503 2580 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 29 14:41:42.216618 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:41:42.216598 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/637fd95f-e250-422b-9132-580af39a1611-cert podName:637fd95f-e250-422b-9132-580af39a1611 nodeName:}" failed. No retries permitted until 2026-07-29 14:41:42.716573845 +0000 UTC m=+379.444723035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/637fd95f-e250-422b-9132-580af39a1611-cert") pod "odh-model-controller-7998c67dc4-mxwh6" (UID: "637fd95f-e250-422b-9132-580af39a1611") : secret "odh-model-controller-webhook-cert" not found Jul 29 14:41:42.224234 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.224216 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq2m5\" (UniqueName: \"kubernetes.io/projected/637fd95f-e250-422b-9132-580af39a1611-kube-api-access-qq2m5\") pod \"odh-model-controller-7998c67dc4-mxwh6\" (UID: \"637fd95f-e250-422b-9132-580af39a1611\") " pod="opendatahub/odh-model-controller-7998c67dc4-mxwh6" Jul 29 14:41:42.234480 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.234458 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68ffcd8f6-t4pfh"] Jul 29 14:41:42.237216 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:41:42.237196 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d59862_3e7f_4c04_9ca7_fe8cacecadf5.slice/crio-665f509e2e9b21e6a366944e2fb41e27e8789f4f3b37c89b48b4ab9e8e6a9fe4 WatchSource:0}: Error finding container 665f509e2e9b21e6a366944e2fb41e27e8789f4f3b37c89b48b4ab9e8e6a9fe4: Status 404 returned error can't find the container with id 665f509e2e9b21e6a366944e2fb41e27e8789f4f3b37c89b48b4ab9e8e6a9fe4 Jul 29 14:41:42.238236 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.238210 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:42.395024 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.394958 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:42.421847 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.421825 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68ffcd8f6-t4pfh" event={"ID":"11d59862-3e7f-4c04-9ca7-fe8cacecadf5","Type":"ContainerStarted","Data":"665f509e2e9b21e6a366944e2fb41e27e8789f4f3b37c89b48b4ab9e8e6a9fe4"} Jul 29 14:41:42.720213 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.720153 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/637fd95f-e250-422b-9132-580af39a1611-cert\") pod \"odh-model-controller-7998c67dc4-mxwh6\" (UID: \"637fd95f-e250-422b-9132-580af39a1611\") " pod="opendatahub/odh-model-controller-7998c67dc4-mxwh6" Jul 29 14:41:42.722521 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.722492 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/637fd95f-e250-422b-9132-580af39a1611-cert\") pod \"odh-model-controller-7998c67dc4-mxwh6\" (UID: \"637fd95f-e250-422b-9132-580af39a1611\") " pod="opendatahub/odh-model-controller-7998c67dc4-mxwh6" Jul 29 14:41:42.788093 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.788059 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:42.943127 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:42.943091 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mxwh6" Jul 29 14:41:43.087198 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.087121 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mxwh6"] Jul 29 14:41:43.087847 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.087818 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:43.091082 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:41:43.091057 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod637fd95f_e250_422b_9132_580af39a1611.slice/crio-9b229c7cbf9776a0a5ab0b930e4b772351396fd5716ee28d10641dc77d8dead7 WatchSource:0}: Error finding container 9b229c7cbf9776a0a5ab0b930e4b772351396fd5716ee28d10641dc77d8dead7: Status 404 returned error can't find the container with id 9b229c7cbf9776a0a5ab0b930e4b772351396fd5716ee28d10641dc77d8dead7 Jul 29 14:41:43.348337 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.348261 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:43.355970 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.355884 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2"] Jul 29 14:41:43.359631 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.359598 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.361953 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.361790 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 29 14:41:43.361953 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.361831 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 29 14:41:43.361953 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.361850 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 29 14:41:43.361953 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.361861 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-68xcd\"" Jul 29 14:41:43.361953 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.361880 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 29 14:41:43.371740 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.371707 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2"] Jul 29 14:41:43.417081 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.417057 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:43.427062 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.427033 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0330ca46-7172-41d4-827e-51e934d9cbde-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-4bdv2\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.427161 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.427083 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0330ca46-7172-41d4-827e-51e934d9cbde-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-4bdv2\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.427161 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.427135 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0330ca46-7172-41d4-827e-51e934d9cbde-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-4bdv2\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.427274 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.427171 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0330ca46-7172-41d4-827e-51e934d9cbde-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-4bdv2\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.427274 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.427257 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0330ca46-7172-41d4-827e-51e934d9cbde-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-4bdv2\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.427371 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.427307 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0330ca46-7172-41d4-827e-51e934d9cbde-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-4bdv2\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.427371 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.427333 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv7vg\" (UniqueName: \"kubernetes.io/projected/0330ca46-7172-41d4-827e-51e934d9cbde-kube-api-access-nv7vg\") pod \"istiod-openshift-gateway-7b7995d5cd-4bdv2\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.431666 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.431629 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mxwh6" event={"ID":"637fd95f-e250-422b-9132-580af39a1611","Type":"ContainerStarted","Data":"9b229c7cbf9776a0a5ab0b930e4b772351396fd5716ee28d10641dc77d8dead7"} Jul 29 14:41:43.528636 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.528609 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0330ca46-7172-41d4-827e-51e934d9cbde-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-4bdv2\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.528802 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.528646 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0330ca46-7172-41d4-827e-51e934d9cbde-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-4bdv2\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.528802 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.528666 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv7vg\" (UniqueName: \"kubernetes.io/projected/0330ca46-7172-41d4-827e-51e934d9cbde-kube-api-access-nv7vg\") pod \"istiod-openshift-gateway-7b7995d5cd-4bdv2\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.528802 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.528702 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0330ca46-7172-41d4-827e-51e934d9cbde-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-4bdv2\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.528802 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.528744 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0330ca46-7172-41d4-827e-51e934d9cbde-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-4bdv2\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.528802 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.528776 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0330ca46-7172-41d4-827e-51e934d9cbde-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-4bdv2\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.528802 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.528801 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0330ca46-7172-41d4-827e-51e934d9cbde-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-4bdv2\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.529386 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.529361 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0330ca46-7172-41d4-827e-51e934d9cbde-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-4bdv2\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.530844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.530824 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0330ca46-7172-41d4-827e-51e934d9cbde-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-4bdv2\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.530844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.530840 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0330ca46-7172-41d4-827e-51e934d9cbde-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-4bdv2\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.531238 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.531216 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0330ca46-7172-41d4-827e-51e934d9cbde-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-4bdv2\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.531323 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.531273 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0330ca46-7172-41d4-827e-51e934d9cbde-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-4bdv2\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.544949 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.544930 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0330ca46-7172-41d4-827e-51e934d9cbde-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-4bdv2\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.545191 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.545174 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv7vg\" (UniqueName: \"kubernetes.io/projected/0330ca46-7172-41d4-827e-51e934d9cbde-kube-api-access-nv7vg\") pod \"istiod-openshift-gateway-7b7995d5cd-4bdv2\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.671305 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.671254 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:43.822353 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:43.822322 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2"] Jul 29 14:41:43.826922 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:41:43.826669 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0330ca46_7172_41d4_827e_51e934d9cbde.slice/crio-b31cb2c99688ebb161b8f7d3f38ca6f757c1dce575af0f0a9ae3eb48672d0b26 WatchSource:0}: Error finding container b31cb2c99688ebb161b8f7d3f38ca6f757c1dce575af0f0a9ae3eb48672d0b26: Status 404 returned error can't find the container with id b31cb2c99688ebb161b8f7d3f38ca6f757c1dce575af0f0a9ae3eb48672d0b26 Jul 29 14:41:44.253513 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:44.253463 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:44.353070 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:44.352982 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:44.416125 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:44.416026 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:44.439952 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:44.439908 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" event={"ID":"0330ca46-7172-41d4-827e-51e934d9cbde","Type":"ContainerStarted","Data":"b31cb2c99688ebb161b8f7d3f38ca6f757c1dce575af0f0a9ae3eb48672d0b26"} Jul 29 14:41:46.452946 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:46.452844 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mxwh6" event={"ID":"637fd95f-e250-422b-9132-580af39a1611","Type":"ContainerStarted","Data":"d696d71805a0d43bab0ae1a1e33ce5343bcd5af8ba834d77932aec29a0b7dcbd"} Jul 29 14:41:46.453358 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:46.452961 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-mxwh6" Jul 29 14:41:46.455195 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:46.455161 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68ffcd8f6-t4pfh" event={"ID":"11d59862-3e7f-4c04-9ca7-fe8cacecadf5","Type":"ContainerStarted","Data":"0877901d9ea3c823d89fa4399bb1777f1c3ea98e133b3e51f60ed6bc866d4e82"} Jul 29 14:41:46.470882 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:46.470830 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-mxwh6" podStartSLOduration=2.3252675529999998 podStartE2EDuration="5.470814139s" totalImagesPullingTime="3.145546586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:41:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:41:46.468769268 +0000 UTC m=+383.196918482" watchObservedRunningTime="2026-07-29 14:41:46.470814139 +0000 UTC m=+383.198963310" Jul 29 14:41:46.486638 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:46.486598 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-68ffcd8f6-t4pfh" podStartSLOduration=1.5316152299999999 podStartE2EDuration="5.4865856s" totalImagesPullingTime="3.95497037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:41:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:41:46.48370346 +0000 UTC m=+383.211852649" watchObservedRunningTime="2026-07-29 14:41:46.4865856 +0000 UTC m=+383.214734791" Jul 29 14:41:47.529823 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:47.529786 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 14:41:47.530176 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:47.529861 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 14:41:48.190505 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:48.190465 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:48.259040 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:48.259009 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:48.465545 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:48.465470 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" event={"ID":"0330ca46-7172-41d4-827e-51e934d9cbde","Type":"ContainerStarted","Data":"034e1a35b0ca48ad526b890ab07baf6494cd3c129f50b289be40e8112e74670c"} Jul 29 14:41:48.465746 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:48.465650 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:48.467380 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:48.467354 2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-4bdv2 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 29 14:41:48.467476 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:48.467401 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" podUID="0330ca46-7172-41d4-827e-51e934d9cbde" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 14:41:48.484253 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:48.484210 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" podStartSLOduration=1.7847180630000001 podStartE2EDuration="5.484193521s" totalImagesPullingTime="3.699475458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:41:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:41:48.482145602 +0000 UTC m=+385.210294792" watchObservedRunningTime="2026-07-29 14:41:48.484193521 +0000 UTC m=+385.212342709" Jul 29 14:41:48.813950 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:48.813885 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:48.911433 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:48.911395 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:49.139872 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:49.139786 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:49.471273 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:49.471205 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:50.868753 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:50.868705 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm"] Jul 29 14:41:50.872265 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:50.872247 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:50.888664 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:50.888640 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm"] Jul 29 14:41:50.993036 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:50.993009 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltzdv\" (UniqueName: \"kubernetes.io/projected/1c84156a-763b-44a5-bbb7-21673d1a9a51-kube-api-access-ltzdv\") pod \"istiod-openshift-gateway-5677ccb49b-kq4pm\" (UID: \"1c84156a-763b-44a5-bbb7-21673d1a9a51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:50.993144 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:50.993053 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c84156a-763b-44a5-bbb7-21673d1a9a51-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-kq4pm\" (UID: \"1c84156a-763b-44a5-bbb7-21673d1a9a51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:50.993144 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:50.993071 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1c84156a-763b-44a5-bbb7-21673d1a9a51-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-kq4pm\" (UID: \"1c84156a-763b-44a5-bbb7-21673d1a9a51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:50.993144 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:50.993124 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1c84156a-763b-44a5-bbb7-21673d1a9a51-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-kq4pm\" (UID: \"1c84156a-763b-44a5-bbb7-21673d1a9a51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:50.993243 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:50.993162 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1c84156a-763b-44a5-bbb7-21673d1a9a51-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-kq4pm\" (UID: \"1c84156a-763b-44a5-bbb7-21673d1a9a51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:50.993243 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:50.993207 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1c84156a-763b-44a5-bbb7-21673d1a9a51-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-kq4pm\" (UID: \"1c84156a-763b-44a5-bbb7-21673d1a9a51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:50.993243 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:50.993237 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c84156a-763b-44a5-bbb7-21673d1a9a51-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-kq4pm\" (UID: \"1c84156a-763b-44a5-bbb7-21673d1a9a51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:51.094411 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.094377 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c84156a-763b-44a5-bbb7-21673d1a9a51-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-kq4pm\" (UID: \"1c84156a-763b-44a5-bbb7-21673d1a9a51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:51.094518 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.094415 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltzdv\" (UniqueName: \"kubernetes.io/projected/1c84156a-763b-44a5-bbb7-21673d1a9a51-kube-api-access-ltzdv\") pod \"istiod-openshift-gateway-5677ccb49b-kq4pm\" (UID: \"1c84156a-763b-44a5-bbb7-21673d1a9a51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:51.094518 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.094457 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c84156a-763b-44a5-bbb7-21673d1a9a51-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-kq4pm\" (UID: \"1c84156a-763b-44a5-bbb7-21673d1a9a51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:51.094518 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.094482 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1c84156a-763b-44a5-bbb7-21673d1a9a51-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-kq4pm\" (UID: \"1c84156a-763b-44a5-bbb7-21673d1a9a51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:51.094518 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.094510 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1c84156a-763b-44a5-bbb7-21673d1a9a51-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-kq4pm\" (UID: \"1c84156a-763b-44a5-bbb7-21673d1a9a51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:51.094712 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.094565 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1c84156a-763b-44a5-bbb7-21673d1a9a51-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-kq4pm\" (UID: \"1c84156a-763b-44a5-bbb7-21673d1a9a51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:51.094712 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.094609 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1c84156a-763b-44a5-bbb7-21673d1a9a51-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-kq4pm\" (UID: \"1c84156a-763b-44a5-bbb7-21673d1a9a51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:51.095348 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.095321 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1c84156a-763b-44a5-bbb7-21673d1a9a51-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-kq4pm\" (UID: \"1c84156a-763b-44a5-bbb7-21673d1a9a51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:51.097055 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.097031 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c84156a-763b-44a5-bbb7-21673d1a9a51-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-kq4pm\" (UID: \"1c84156a-763b-44a5-bbb7-21673d1a9a51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:51.097055 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.097039 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1c84156a-763b-44a5-bbb7-21673d1a9a51-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-kq4pm\" (UID: \"1c84156a-763b-44a5-bbb7-21673d1a9a51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:51.097216 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.097197 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1c84156a-763b-44a5-bbb7-21673d1a9a51-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-kq4pm\" (UID: \"1c84156a-763b-44a5-bbb7-21673d1a9a51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:51.097391 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.097372 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1c84156a-763b-44a5-bbb7-21673d1a9a51-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-kq4pm\" (UID: \"1c84156a-763b-44a5-bbb7-21673d1a9a51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:51.115396 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.115375 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c84156a-763b-44a5-bbb7-21673d1a9a51-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-kq4pm\" (UID: \"1c84156a-763b-44a5-bbb7-21673d1a9a51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:51.117622 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.117601 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltzdv\" (UniqueName: \"kubernetes.io/projected/1c84156a-763b-44a5-bbb7-21673d1a9a51-kube-api-access-ltzdv\") pod \"istiod-openshift-gateway-5677ccb49b-kq4pm\" (UID: \"1c84156a-763b-44a5-bbb7-21673d1a9a51\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:51.181850 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.181772 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:51.310444 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.310415 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm"] Jul 29 14:41:51.313971 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:41:51.313941 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c84156a_763b_44a5_bbb7_21673d1a9a51.slice/crio-c7c7b4b1fad08b2bf2e189031a82880c30df0a43df3f542bccffc0ba8deed655 WatchSource:0}: Error finding container c7c7b4b1fad08b2bf2e189031a82880c30df0a43df3f542bccffc0ba8deed655: Status 404 returned error can't find the container with id c7c7b4b1fad08b2bf2e189031a82880c30df0a43df3f542bccffc0ba8deed655 Jul 29 14:41:51.316844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.316815 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 14:41:51.316904 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.316876 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 29 14:41:51.481612 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.481522 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" event={"ID":"1c84156a-763b-44a5-bbb7-21673d1a9a51","Type":"ContainerStarted","Data":"45fab716ed9398e3cf6b2907cc58845320495a80323b5874e77f366e132fb576"} Jul 29 14:41:51.481612 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.481577 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" event={"ID":"1c84156a-763b-44a5-bbb7-21673d1a9a51","Type":"ContainerStarted","Data":"c7c7b4b1fad08b2bf2e189031a82880c30df0a43df3f542bccffc0ba8deed655"} Jul 29 14:41:51.481808 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.481737 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:51.506520 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:51.506461 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" podStartSLOduration=1.506446155 podStartE2EDuration="1.506446155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:41:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:41:51.505409628 +0000 UTC m=+388.233558907" watchObservedRunningTime="2026-07-29 14:41:51.506446155 +0000 UTC m=+388.234595345" Jul 29 14:41:52.488440 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:52.488412 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-kq4pm" Jul 29 14:41:52.637802 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:52.637772 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2"] Jul 29 14:41:52.637962 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:52.637926 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" podUID="0330ca46-7172-41d4-827e-51e934d9cbde" containerName="discovery" containerID="cri-o://034e1a35b0ca48ad526b890ab07baf6494cd3c129f50b289be40e8112e74670c" gracePeriod=30 Jul 29 14:41:52.875418 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:52.875387 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:52.885744 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:52.885711 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:53.011659 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.011590 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nv7vg\" (UniqueName: \"kubernetes.io/projected/0330ca46-7172-41d4-827e-51e934d9cbde-kube-api-access-nv7vg\") pod \"0330ca46-7172-41d4-827e-51e934d9cbde\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " Jul 29 14:41:53.011659 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.011633 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0330ca46-7172-41d4-827e-51e934d9cbde-istio-token\") pod \"0330ca46-7172-41d4-827e-51e934d9cbde\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " Jul 29 14:41:53.011659 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.011660 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0330ca46-7172-41d4-827e-51e934d9cbde-local-certs\") pod \"0330ca46-7172-41d4-827e-51e934d9cbde\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " Jul 29 14:41:53.011912 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.011686 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0330ca46-7172-41d4-827e-51e934d9cbde-cacerts\") pod \"0330ca46-7172-41d4-827e-51e934d9cbde\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " Jul 29 14:41:53.011912 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.011739 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0330ca46-7172-41d4-827e-51e934d9cbde-istio-kubeconfig\") pod \"0330ca46-7172-41d4-827e-51e934d9cbde\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " Jul 29 14:41:53.011912 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.011791 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0330ca46-7172-41d4-827e-51e934d9cbde-istio-csr-ca-configmap\") pod \"0330ca46-7172-41d4-827e-51e934d9cbde\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " Jul 29 14:41:53.011912 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.011817 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0330ca46-7172-41d4-827e-51e934d9cbde-istio-csr-dns-cert\") pod \"0330ca46-7172-41d4-827e-51e934d9cbde\" (UID: \"0330ca46-7172-41d4-827e-51e934d9cbde\") " Jul 29 14:41:53.012238 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.012194 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0330ca46-7172-41d4-827e-51e934d9cbde-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "0330ca46-7172-41d4-827e-51e934d9cbde" (UID: "0330ca46-7172-41d4-827e-51e934d9cbde"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 14:41:53.014559 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.014530 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0330ca46-7172-41d4-827e-51e934d9cbde-kube-api-access-nv7vg" (OuterVolumeSpecName: "kube-api-access-nv7vg") pod "0330ca46-7172-41d4-827e-51e934d9cbde" (UID: "0330ca46-7172-41d4-827e-51e934d9cbde"). InnerVolumeSpecName "kube-api-access-nv7vg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 14:41:53.014680 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.014651 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0330ca46-7172-41d4-827e-51e934d9cbde-local-certs" (OuterVolumeSpecName: "local-certs") pod "0330ca46-7172-41d4-827e-51e934d9cbde" (UID: "0330ca46-7172-41d4-827e-51e934d9cbde"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:41:53.014796 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.014658 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0330ca46-7172-41d4-827e-51e934d9cbde-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "0330ca46-7172-41d4-827e-51e934d9cbde" (UID: "0330ca46-7172-41d4-827e-51e934d9cbde"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 14:41:53.014796 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.014752 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0330ca46-7172-41d4-827e-51e934d9cbde-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "0330ca46-7172-41d4-827e-51e934d9cbde" (UID: "0330ca46-7172-41d4-827e-51e934d9cbde"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 14:41:53.014896 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.014791 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0330ca46-7172-41d4-827e-51e934d9cbde-cacerts" (OuterVolumeSpecName: "cacerts") pod "0330ca46-7172-41d4-827e-51e934d9cbde" (UID: "0330ca46-7172-41d4-827e-51e934d9cbde"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 14:41:53.015017 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.014994 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0330ca46-7172-41d4-827e-51e934d9cbde-istio-token" (OuterVolumeSpecName: "istio-token") pod "0330ca46-7172-41d4-827e-51e934d9cbde" (UID: "0330ca46-7172-41d4-827e-51e934d9cbde"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 14:41:53.038172 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.038147 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-674rf"] Jul 29 14:41:53.038473 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.038460 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0330ca46-7172-41d4-827e-51e934d9cbde" containerName="discovery" Jul 29 14:41:53.038519 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.038476 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0330ca46-7172-41d4-827e-51e934d9cbde" containerName="discovery" Jul 29 14:41:53.038555 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.038540 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0330ca46-7172-41d4-827e-51e934d9cbde" containerName="discovery" Jul 29 14:41:53.041483 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.041468 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-f8ljt"] Jul 29 14:41:53.041594 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.041578 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-674rf" Jul 29 14:41:53.043821 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.043805 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-fsbx5\"" Jul 29 14:41:53.043888 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.043820 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 29 14:41:53.044449 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.044435 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f8ljt" Jul 29 14:41:53.050169 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.050154 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 29 14:41:53.050810 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.050796 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-2zbnn\"" Jul 29 14:41:53.062139 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.062112 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:53.066593 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.063274 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-674rf"] Jul 29 14:41:53.070792 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.070758 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-f8ljt"] Jul 29 14:41:53.113152 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.113126 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kch9v\" (UniqueName: \"kubernetes.io/projected/186209d4-1983-42d8-80d8-246f3a7b3fad-kube-api-access-kch9v\") pod \"kserve-controller-manager-d6f4b65b8-f8ljt\" (UID: \"186209d4-1983-42d8-80d8-246f3a7b3fad\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f8ljt" Jul 29 14:41:53.113255 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.113175 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/186209d4-1983-42d8-80d8-246f3a7b3fad-cert\") pod \"kserve-controller-manager-d6f4b65b8-f8ljt\" (UID: \"186209d4-1983-42d8-80d8-246f3a7b3fad\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f8ljt" Jul 29 14:41:53.113315 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.113250 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/35a7955b-5872-44c6-9237-62a0f95fe24c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-674rf\" (UID: \"35a7955b-5872-44c6-9237-62a0f95fe24c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-674rf" Jul 29 14:41:53.113315 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.113292 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd492\" (UniqueName: \"kubernetes.io/projected/35a7955b-5872-44c6-9237-62a0f95fe24c-kube-api-access-wd492\") pod \"llmisvc-controller-manager-777c6bd7d5-674rf\" (UID: \"35a7955b-5872-44c6-9237-62a0f95fe24c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-674rf" Jul 29 14:41:53.113390 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.113325 2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0330ca46-7172-41d4-827e-51e934d9cbde-istio-csr-ca-configmap\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:41:53.113390 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.113335 2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0330ca46-7172-41d4-827e-51e934d9cbde-istio-csr-dns-cert\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:41:53.113390 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.113344 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nv7vg\" (UniqueName: \"kubernetes.io/projected/0330ca46-7172-41d4-827e-51e934d9cbde-kube-api-access-nv7vg\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:41:53.113390 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.113352 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0330ca46-7172-41d4-827e-51e934d9cbde-istio-token\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:41:53.113390 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.113360 2580 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0330ca46-7172-41d4-827e-51e934d9cbde-local-certs\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:41:53.113390 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.113367 2580 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0330ca46-7172-41d4-827e-51e934d9cbde-cacerts\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:41:53.113390 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.113374 2580 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0330ca46-7172-41d4-827e-51e934d9cbde-istio-kubeconfig\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:41:53.214685 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.214659 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/186209d4-1983-42d8-80d8-246f3a7b3fad-cert\") pod \"kserve-controller-manager-d6f4b65b8-f8ljt\" (UID: \"186209d4-1983-42d8-80d8-246f3a7b3fad\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f8ljt" Jul 29 14:41:53.214814 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.214753 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/35a7955b-5872-44c6-9237-62a0f95fe24c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-674rf\" (UID: \"35a7955b-5872-44c6-9237-62a0f95fe24c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-674rf" Jul 29 14:41:53.214814 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.214784 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd492\" (UniqueName: \"kubernetes.io/projected/35a7955b-5872-44c6-9237-62a0f95fe24c-kube-api-access-wd492\") pod \"llmisvc-controller-manager-777c6bd7d5-674rf\" (UID: \"35a7955b-5872-44c6-9237-62a0f95fe24c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-674rf" Jul 29 14:41:53.214920 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:41:53.214807 2580 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 29 14:41:53.214920 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:41:53.214859 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/186209d4-1983-42d8-80d8-246f3a7b3fad-cert podName:186209d4-1983-42d8-80d8-246f3a7b3fad nodeName:}" failed. No retries permitted until 2026-07-29 14:41:53.71484403 +0000 UTC m=+390.442993197 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/186209d4-1983-42d8-80d8-246f3a7b3fad-cert") pod "kserve-controller-manager-d6f4b65b8-f8ljt" (UID: "186209d4-1983-42d8-80d8-246f3a7b3fad") : secret "kserve-webhook-server-cert" not found Jul 29 14:41:53.215023 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.214927 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kch9v\" (UniqueName: \"kubernetes.io/projected/186209d4-1983-42d8-80d8-246f3a7b3fad-kube-api-access-kch9v\") pod \"kserve-controller-manager-d6f4b65b8-f8ljt\" (UID: \"186209d4-1983-42d8-80d8-246f3a7b3fad\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f8ljt" Jul 29 14:41:53.217111 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.217092 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/35a7955b-5872-44c6-9237-62a0f95fe24c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-674rf\" (UID: \"35a7955b-5872-44c6-9237-62a0f95fe24c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-674rf" Jul 29 14:41:53.233392 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.233367 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kch9v\" (UniqueName: \"kubernetes.io/projected/186209d4-1983-42d8-80d8-246f3a7b3fad-kube-api-access-kch9v\") pod \"kserve-controller-manager-d6f4b65b8-f8ljt\" (UID: \"186209d4-1983-42d8-80d8-246f3a7b3fad\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f8ljt" Jul 29 14:41:53.233637 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.233620 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd492\" (UniqueName: \"kubernetes.io/projected/35a7955b-5872-44c6-9237-62a0f95fe24c-kube-api-access-wd492\") pod \"llmisvc-controller-manager-777c6bd7d5-674rf\" (UID: \"35a7955b-5872-44c6-9237-62a0f95fe24c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-674rf" Jul 29 14:41:53.270150 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.270085 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:53.351912 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.351887 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-674rf" Jul 29 14:41:53.439432 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.439403 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:53.491873 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.491846 2580 generic.go:356] "Generic (PLEG): container finished" podID="0330ca46-7172-41d4-827e-51e934d9cbde" containerID="034e1a35b0ca48ad526b890ab07baf6494cd3c129f50b289be40e8112e74670c" exitCode=0 Jul 29 14:41:53.492284 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.491904 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" Jul 29 14:41:53.492284 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.491927 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" event={"ID":"0330ca46-7172-41d4-827e-51e934d9cbde","Type":"ContainerDied","Data":"034e1a35b0ca48ad526b890ab07baf6494cd3c129f50b289be40e8112e74670c"} Jul 29 14:41:53.492284 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.491984 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2" event={"ID":"0330ca46-7172-41d4-827e-51e934d9cbde","Type":"ContainerDied","Data":"b31cb2c99688ebb161b8f7d3f38ca6f757c1dce575af0f0a9ae3eb48672d0b26"} Jul 29 14:41:53.492284 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.492015 2580 scope.go:117] "RemoveContainer" containerID="034e1a35b0ca48ad526b890ab07baf6494cd3c129f50b289be40e8112e74670c" Jul 29 14:41:53.500905 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.500885 2580 scope.go:117] "RemoveContainer" containerID="034e1a35b0ca48ad526b890ab07baf6494cd3c129f50b289be40e8112e74670c" Jul 29 14:41:53.501401 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:41:53.501378 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"034e1a35b0ca48ad526b890ab07baf6494cd3c129f50b289be40e8112e74670c\": container with ID starting with 034e1a35b0ca48ad526b890ab07baf6494cd3c129f50b289be40e8112e74670c not found: ID does not exist" containerID="034e1a35b0ca48ad526b890ab07baf6494cd3c129f50b289be40e8112e74670c" Jul 29 14:41:53.501486 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.501407 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"034e1a35b0ca48ad526b890ab07baf6494cd3c129f50b289be40e8112e74670c"} err="failed to get container status \"034e1a35b0ca48ad526b890ab07baf6494cd3c129f50b289be40e8112e74670c\": rpc error: code = NotFound desc = could not find container \"034e1a35b0ca48ad526b890ab07baf6494cd3c129f50b289be40e8112e74670c\": container with ID starting with 034e1a35b0ca48ad526b890ab07baf6494cd3c129f50b289be40e8112e74670c not found: ID does not exist" Jul 29 14:41:53.503293 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.503277 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-674rf"] Jul 29 14:41:53.504647 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:41:53.504626 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod35a7955b_5872_44c6_9237_62a0f95fe24c.slice/crio-d356413631c2c5e8b8e26e74b80baea5b89f19d3d2ca04aebda446e3818402ae WatchSource:0}: Error finding container d356413631c2c5e8b8e26e74b80baea5b89f19d3d2ca04aebda446e3818402ae: Status 404 returned error can't find the container with id d356413631c2c5e8b8e26e74b80baea5b89f19d3d2ca04aebda446e3818402ae Jul 29 14:41:53.522940 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.522895 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2"] Jul 29 14:41:53.529364 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.529347 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-4bdv2"] Jul 29 14:41:53.718697 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.718660 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/186209d4-1983-42d8-80d8-246f3a7b3fad-cert\") pod \"kserve-controller-manager-d6f4b65b8-f8ljt\" (UID: \"186209d4-1983-42d8-80d8-246f3a7b3fad\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f8ljt" Jul 29 14:41:53.720907 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.720883 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/186209d4-1983-42d8-80d8-246f3a7b3fad-cert\") pod \"kserve-controller-manager-d6f4b65b8-f8ljt\" (UID: \"186209d4-1983-42d8-80d8-246f3a7b3fad\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f8ljt" Jul 29 14:41:53.816851 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.816790 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0330ca46-7172-41d4-827e-51e934d9cbde" path="/var/lib/kubelet/pods/0330ca46-7172-41d4-827e-51e934d9cbde/volumes" Jul 29 14:41:53.957561 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:53.957538 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f8ljt" Jul 29 14:41:54.098115 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:54.098091 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-f8ljt"] Jul 29 14:41:54.100214 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:41:54.100186 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod186209d4_1983_42d8_80d8_246f3a7b3fad.slice/crio-e490be1dd3485308578f44e803413383b4619510f93f3604248661aecbcfdaf4 WatchSource:0}: Error finding container e490be1dd3485308578f44e803413383b4619510f93f3604248661aecbcfdaf4: Status 404 returned error can't find the container with id e490be1dd3485308578f44e803413383b4619510f93f3604248661aecbcfdaf4 Jul 29 14:41:54.215827 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:54.215793 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:54.418836 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:54.418757 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:54.497660 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:54.497622 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-674rf" event={"ID":"35a7955b-5872-44c6-9237-62a0f95fe24c","Type":"ContainerStarted","Data":"d356413631c2c5e8b8e26e74b80baea5b89f19d3d2ca04aebda446e3818402ae"} Jul 29 14:41:54.498940 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:54.498913 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f8ljt" event={"ID":"186209d4-1983-42d8-80d8-246f3a7b3fad","Type":"ContainerStarted","Data":"e490be1dd3485308578f44e803413383b4619510f93f3604248661aecbcfdaf4"} Jul 29 14:41:54.620224 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:54.620192 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:41:57.461795 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:57.461712 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-mxwh6" Jul 29 14:41:57.516161 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:57.516128 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-674rf" event={"ID":"35a7955b-5872-44c6-9237-62a0f95fe24c","Type":"ContainerStarted","Data":"38e25913ba443d8bef2008c11422f9025f630cef51c15ff480c0d6a342836dfa"} Jul 29 14:41:57.516343 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:57.516189 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-674rf" Jul 29 14:41:57.517496 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:57.517467 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f8ljt" event={"ID":"186209d4-1983-42d8-80d8-246f3a7b3fad","Type":"ContainerStarted","Data":"91e78c1488a3b867ed25241958faf94b314d37ac4ef032104905ffb33ae09df1"} Jul 29 14:41:57.517635 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:57.517618 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f8ljt" Jul 29 14:41:57.531873 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:57.531836 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-674rf" podStartSLOduration=1.889276389 podStartE2EDuration="5.531822973s" totalImagesPullingTime="3.642546584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:41:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:41:57.530717648 +0000 UTC m=+394.258866852" watchObservedRunningTime="2026-07-29 14:41:57.531822973 +0000 UTC m=+394.259972190" Jul 29 14:41:57.545604 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:41:57.545566 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f8ljt" podStartSLOduration=1.496080936 podStartE2EDuration="4.54555351s" totalImagesPullingTime="3.049472574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:41:57.545079165 +0000 UTC m=+394.273228354" watchObservedRunningTime="2026-07-29 14:41:57.54555351 +0000 UTC m=+394.273702699" Jul 29 14:42:28.524694 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:28.524657 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-674rf" Jul 29 14:42:28.527698 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:28.527678 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f8ljt" Jul 29 14:42:34.197535 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:34.197497 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:42:34.262977 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:34.262945 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:42:42.470463 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:42.470418 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:42:42.570265 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:42.570231 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:42:42.654837 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:42.654803 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:42:43.411077 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:43.411033 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:42:43.611119 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:43.611091 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:42:52.028580 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:52.028470 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:42:54.598799 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:54.598757 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s"] Jul 29 14:42:54.601677 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:54.601657 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s" Jul 29 14:42:54.603818 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:54.603787 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bvthn\"" Jul 29 14:42:54.603980 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:54.603960 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 29 14:42:54.604220 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:54.604206 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 29 14:42:54.610516 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:54.610489 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s"] Jul 29 14:42:54.673073 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:54.673043 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxq6b\" (UniqueName: \"kubernetes.io/projected/8c3f17c4-f6be-4645-b934-61a6a5499b6d-kube-api-access-mxq6b\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s\" (UID: \"8c3f17c4-f6be-4645-b934-61a6a5499b6d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s" Jul 29 14:42:54.673172 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:54.673078 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c3f17c4-f6be-4645-b934-61a6a5499b6d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s\" (UID: \"8c3f17c4-f6be-4645-b934-61a6a5499b6d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s" Jul 29 14:42:54.673214 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:54.673170 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c3f17c4-f6be-4645-b934-61a6a5499b6d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s\" (UID: \"8c3f17c4-f6be-4645-b934-61a6a5499b6d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s" Jul 29 14:42:54.774269 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:54.774240 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxq6b\" (UniqueName: \"kubernetes.io/projected/8c3f17c4-f6be-4645-b934-61a6a5499b6d-kube-api-access-mxq6b\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s\" (UID: \"8c3f17c4-f6be-4645-b934-61a6a5499b6d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s" Jul 29 14:42:54.774364 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:54.774281 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c3f17c4-f6be-4645-b934-61a6a5499b6d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s\" (UID: \"8c3f17c4-f6be-4645-b934-61a6a5499b6d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s" Jul 29 14:42:54.774364 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:54.774314 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c3f17c4-f6be-4645-b934-61a6a5499b6d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s\" (UID: \"8c3f17c4-f6be-4645-b934-61a6a5499b6d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s" Jul 29 14:42:54.774700 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:54.774684 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c3f17c4-f6be-4645-b934-61a6a5499b6d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s\" (UID: \"8c3f17c4-f6be-4645-b934-61a6a5499b6d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s" Jul 29 14:42:54.774765 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:54.774712 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c3f17c4-f6be-4645-b934-61a6a5499b6d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s\" (UID: \"8c3f17c4-f6be-4645-b934-61a6a5499b6d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s" Jul 29 14:42:54.781910 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:54.781880 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxq6b\" (UniqueName: \"kubernetes.io/projected/8c3f17c4-f6be-4645-b934-61a6a5499b6d-kube-api-access-mxq6b\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s\" (UID: \"8c3f17c4-f6be-4645-b934-61a6a5499b6d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s" Jul 29 14:42:54.912291 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:54.912241 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s" Jul 29 14:42:55.039248 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.039217 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s"] Jul 29 14:42:55.042971 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:42:55.042943 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c3f17c4_f6be_4645_b934_61a6a5499b6d.slice/crio-0e1ee9e1a13dedb926a66ea7863589ac5ebc94f3ae73555e161b3b201d148310 WatchSource:0}: Error finding container 0e1ee9e1a13dedb926a66ea7863589ac5ebc94f3ae73555e161b3b201d148310: Status 404 returned error can't find the container with id 0e1ee9e1a13dedb926a66ea7863589ac5ebc94f3ae73555e161b3b201d148310 Jul 29 14:42:55.198631 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.198578 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq"] Jul 29 14:42:55.201508 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.201493 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq" Jul 29 14:42:55.210302 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.210283 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq"] Jul 29 14:42:55.279036 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.279017 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d060c49-d966-486c-bf4e-35b6c3573e0a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq\" (UID: \"4d060c49-d966-486c-bf4e-35b6c3573e0a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq" Jul 29 14:42:55.279139 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.279066 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q457b\" (UniqueName: \"kubernetes.io/projected/4d060c49-d966-486c-bf4e-35b6c3573e0a-kube-api-access-q457b\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq\" (UID: \"4d060c49-d966-486c-bf4e-35b6c3573e0a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq" Jul 29 14:42:55.279188 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.279159 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d060c49-d966-486c-bf4e-35b6c3573e0a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq\" (UID: \"4d060c49-d966-486c-bf4e-35b6c3573e0a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq" Jul 29 14:42:55.379742 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.379700 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d060c49-d966-486c-bf4e-35b6c3573e0a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq\" (UID: \"4d060c49-d966-486c-bf4e-35b6c3573e0a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq" Jul 29 14:42:55.379828 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.379766 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q457b\" (UniqueName: \"kubernetes.io/projected/4d060c49-d966-486c-bf4e-35b6c3573e0a-kube-api-access-q457b\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq\" (UID: \"4d060c49-d966-486c-bf4e-35b6c3573e0a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq" Jul 29 14:42:55.379869 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.379838 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d060c49-d966-486c-bf4e-35b6c3573e0a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq\" (UID: \"4d060c49-d966-486c-bf4e-35b6c3573e0a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq" Jul 29 14:42:55.380109 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.380089 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d060c49-d966-486c-bf4e-35b6c3573e0a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq\" (UID: \"4d060c49-d966-486c-bf4e-35b6c3573e0a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq" Jul 29 14:42:55.380144 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.380122 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d060c49-d966-486c-bf4e-35b6c3573e0a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq\" (UID: \"4d060c49-d966-486c-bf4e-35b6c3573e0a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq" Jul 29 14:42:55.387810 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.387785 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q457b\" (UniqueName: \"kubernetes.io/projected/4d060c49-d966-486c-bf4e-35b6c3573e0a-kube-api-access-q457b\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq\" (UID: \"4d060c49-d966-486c-bf4e-35b6c3573e0a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq" Jul 29 14:42:55.510572 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.510552 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq" Jul 29 14:42:55.633743 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.633693 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq"] Jul 29 14:42:55.635830 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:42:55.635807 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d060c49_d966_486c_bf4e_35b6c3573e0a.slice/crio-cc27f697be4090f314669c68c1414cff004ccc5edd138c3f614ce33172549add WatchSource:0}: Error finding container cc27f697be4090f314669c68c1414cff004ccc5edd138c3f614ce33172549add: Status 404 returned error can't find the container with id cc27f697be4090f314669c68c1414cff004ccc5edd138c3f614ce33172549add Jul 29 14:42:55.801449 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.801427 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls"] Jul 29 14:42:55.804621 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.804607 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls" Jul 29 14:42:55.814884 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.814859 2580 generic.go:356] "Generic (PLEG): container finished" podID="4d060c49-d966-486c-bf4e-35b6c3573e0a" containerID="3671b967a0d2ffcbb195217f5bf802b31c91ce49a2a3c55340b5d86384a951c4" exitCode=0 Jul 29 14:42:55.816969 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.816950 2580 generic.go:356] "Generic (PLEG): container finished" podID="8c3f17c4-f6be-4645-b934-61a6a5499b6d" containerID="6a5adbf3278cbc59b48f41b0189ec29b157b2ed71156d22c738f3c9bcfeb0571" exitCode=0 Jul 29 14:42:55.818975 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.818956 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls"] Jul 29 14:42:55.819049 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.818999 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq" event={"ID":"4d060c49-d966-486c-bf4e-35b6c3573e0a","Type":"ContainerDied","Data":"3671b967a0d2ffcbb195217f5bf802b31c91ce49a2a3c55340b5d86384a951c4"} Jul 29 14:42:55.819049 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.819023 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq" event={"ID":"4d060c49-d966-486c-bf4e-35b6c3573e0a","Type":"ContainerStarted","Data":"cc27f697be4090f314669c68c1414cff004ccc5edd138c3f614ce33172549add"} Jul 29 14:42:55.819049 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.819035 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s" event={"ID":"8c3f17c4-f6be-4645-b934-61a6a5499b6d","Type":"ContainerDied","Data":"6a5adbf3278cbc59b48f41b0189ec29b157b2ed71156d22c738f3c9bcfeb0571"} Jul 29 14:42:55.819049 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.819050 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s" event={"ID":"8c3f17c4-f6be-4645-b934-61a6a5499b6d","Type":"ContainerStarted","Data":"0e1ee9e1a13dedb926a66ea7863589ac5ebc94f3ae73555e161b3b201d148310"} Jul 29 14:42:55.883801 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.883761 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/751fa8fe-b9a5-4026-bd77-f38f73957032-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls\" (UID: \"751fa8fe-b9a5-4026-bd77-f38f73957032\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls" Jul 29 14:42:55.883911 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.883885 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/751fa8fe-b9a5-4026-bd77-f38f73957032-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls\" (UID: \"751fa8fe-b9a5-4026-bd77-f38f73957032\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls" Jul 29 14:42:55.884016 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.883926 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgr7l\" (UniqueName: \"kubernetes.io/projected/751fa8fe-b9a5-4026-bd77-f38f73957032-kube-api-access-bgr7l\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls\" (UID: \"751fa8fe-b9a5-4026-bd77-f38f73957032\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls" Jul 29 14:42:55.984660 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.984637 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/751fa8fe-b9a5-4026-bd77-f38f73957032-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls\" (UID: \"751fa8fe-b9a5-4026-bd77-f38f73957032\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls" Jul 29 14:42:55.984776 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.984674 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgr7l\" (UniqueName: \"kubernetes.io/projected/751fa8fe-b9a5-4026-bd77-f38f73957032-kube-api-access-bgr7l\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls\" (UID: \"751fa8fe-b9a5-4026-bd77-f38f73957032\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls" Jul 29 14:42:55.984776 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.984708 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/751fa8fe-b9a5-4026-bd77-f38f73957032-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls\" (UID: \"751fa8fe-b9a5-4026-bd77-f38f73957032\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls" Jul 29 14:42:55.985039 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.985018 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/751fa8fe-b9a5-4026-bd77-f38f73957032-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls\" (UID: \"751fa8fe-b9a5-4026-bd77-f38f73957032\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls" Jul 29 14:42:55.985100 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.985043 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/751fa8fe-b9a5-4026-bd77-f38f73957032-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls\" (UID: \"751fa8fe-b9a5-4026-bd77-f38f73957032\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls" Jul 29 14:42:55.993009 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:55.992994 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgr7l\" (UniqueName: \"kubernetes.io/projected/751fa8fe-b9a5-4026-bd77-f38f73957032-kube-api-access-bgr7l\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls\" (UID: \"751fa8fe-b9a5-4026-bd77-f38f73957032\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls" Jul 29 14:42:56.114695 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:56.114635 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls" Jul 29 14:42:56.240649 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:56.239193 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls"] Jul 29 14:42:56.392918 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:56.392859 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp"] Jul 29 14:42:56.396117 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:56.396102 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp" Jul 29 14:42:56.403924 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:56.403903 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp"] Jul 29 14:42:56.488852 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:56.488832 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7936a6a4-72ef-49fe-b75d-48e925e0e807-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp\" (UID: \"7936a6a4-72ef-49fe-b75d-48e925e0e807\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp" Jul 29 14:42:56.488952 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:56.488861 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx9km\" (UniqueName: \"kubernetes.io/projected/7936a6a4-72ef-49fe-b75d-48e925e0e807-kube-api-access-gx9km\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp\" (UID: \"7936a6a4-72ef-49fe-b75d-48e925e0e807\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp" Jul 29 14:42:56.488952 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:56.488893 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7936a6a4-72ef-49fe-b75d-48e925e0e807-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp\" (UID: \"7936a6a4-72ef-49fe-b75d-48e925e0e807\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp" Jul 29 14:42:56.589844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:56.589819 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7936a6a4-72ef-49fe-b75d-48e925e0e807-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp\" (UID: \"7936a6a4-72ef-49fe-b75d-48e925e0e807\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp" Jul 29 14:42:56.589952 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:56.589857 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx9km\" (UniqueName: \"kubernetes.io/projected/7936a6a4-72ef-49fe-b75d-48e925e0e807-kube-api-access-gx9km\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp\" (UID: \"7936a6a4-72ef-49fe-b75d-48e925e0e807\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp" Jul 29 14:42:56.589952 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:56.589903 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7936a6a4-72ef-49fe-b75d-48e925e0e807-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp\" (UID: \"7936a6a4-72ef-49fe-b75d-48e925e0e807\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp" Jul 29 14:42:56.590137 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:56.590117 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7936a6a4-72ef-49fe-b75d-48e925e0e807-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp\" (UID: \"7936a6a4-72ef-49fe-b75d-48e925e0e807\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp" Jul 29 14:42:56.590230 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:56.590211 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7936a6a4-72ef-49fe-b75d-48e925e0e807-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp\" (UID: \"7936a6a4-72ef-49fe-b75d-48e925e0e807\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp" Jul 29 14:42:56.597877 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:56.597855 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx9km\" (UniqueName: \"kubernetes.io/projected/7936a6a4-72ef-49fe-b75d-48e925e0e807-kube-api-access-gx9km\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp\" (UID: \"7936a6a4-72ef-49fe-b75d-48e925e0e807\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp" Jul 29 14:42:56.705275 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:56.705220 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp" Jul 29 14:42:56.823986 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:56.823912 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls" event={"ID":"751fa8fe-b9a5-4026-bd77-f38f73957032","Type":"ContainerDied","Data":"63344873a9d89d5a00cf75e2b5358f475cffefdf248b02562bbf20804a60a7f9"} Jul 29 14:42:56.824469 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:56.824396 2580 generic.go:356] "Generic (PLEG): container finished" podID="751fa8fe-b9a5-4026-bd77-f38f73957032" containerID="63344873a9d89d5a00cf75e2b5358f475cffefdf248b02562bbf20804a60a7f9" exitCode=0 Jul 29 14:42:56.824564 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:56.824520 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls" event={"ID":"751fa8fe-b9a5-4026-bd77-f38f73957032","Type":"ContainerStarted","Data":"7985bddecc0058ea1e1283741e95324ed44924942f14a629ff0f7d11c0127f1a"} Jul 29 14:42:56.838665 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:56.838640 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp"] Jul 29 14:42:56.841950 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:42:56.841921 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7936a6a4_72ef_49fe_b75d_48e925e0e807.slice/crio-103149f373d78e605c63daaae5547335d8cd9ea1f8796777f32cffac661e4fa2 WatchSource:0}: Error finding container 103149f373d78e605c63daaae5547335d8cd9ea1f8796777f32cffac661e4fa2: Status 404 returned error can't find the container with id 103149f373d78e605c63daaae5547335d8cd9ea1f8796777f32cffac661e4fa2 Jul 29 14:42:57.830970 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:57.830942 2580 generic.go:356] "Generic (PLEG): container finished" podID="4d060c49-d966-486c-bf4e-35b6c3573e0a" containerID="78eb61dd5d831eee7903e74e2e57f8ab7dfe42e57ddd2743bf57b135a0f8c660" exitCode=0 Jul 29 14:42:57.831325 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:57.831033 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq" event={"ID":"4d060c49-d966-486c-bf4e-35b6c3573e0a","Type":"ContainerDied","Data":"78eb61dd5d831eee7903e74e2e57f8ab7dfe42e57ddd2743bf57b135a0f8c660"} Jul 29 14:42:57.835694 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:57.835671 2580 generic.go:356] "Generic (PLEG): container finished" podID="8c3f17c4-f6be-4645-b934-61a6a5499b6d" containerID="c39f682988e4bfe037bff39d7ed615b9062c52514e46ee909e29e212a0179f9a" exitCode=0 Jul 29 14:42:57.835798 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:57.835752 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s" event={"ID":"8c3f17c4-f6be-4645-b934-61a6a5499b6d","Type":"ContainerDied","Data":"c39f682988e4bfe037bff39d7ed615b9062c52514e46ee909e29e212a0179f9a"} Jul 29 14:42:57.837437 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:57.837417 2580 generic.go:356] "Generic (PLEG): container finished" podID="751fa8fe-b9a5-4026-bd77-f38f73957032" containerID="2a74f72c5f9ce67b1b18dc718d3ff4a25925f467fbf7cdf80b03d831ebc8d4e0" exitCode=0 Jul 29 14:42:57.837509 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:57.837460 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls" event={"ID":"751fa8fe-b9a5-4026-bd77-f38f73957032","Type":"ContainerDied","Data":"2a74f72c5f9ce67b1b18dc718d3ff4a25925f467fbf7cdf80b03d831ebc8d4e0"} Jul 29 14:42:57.839054 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:57.839034 2580 generic.go:356] "Generic (PLEG): container finished" podID="7936a6a4-72ef-49fe-b75d-48e925e0e807" containerID="e826a94a93e424b099ceab702c2230f96ac32642df2de422774eb29281434c5f" exitCode=0 Jul 29 14:42:57.839123 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:57.839074 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp" event={"ID":"7936a6a4-72ef-49fe-b75d-48e925e0e807","Type":"ContainerDied","Data":"e826a94a93e424b099ceab702c2230f96ac32642df2de422774eb29281434c5f"} Jul 29 14:42:57.839158 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:57.839128 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp" event={"ID":"7936a6a4-72ef-49fe-b75d-48e925e0e807","Type":"ContainerStarted","Data":"103149f373d78e605c63daaae5547335d8cd9ea1f8796777f32cffac661e4fa2"} Jul 29 14:42:58.846253 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:58.846224 2580 generic.go:356] "Generic (PLEG): container finished" podID="4d060c49-d966-486c-bf4e-35b6c3573e0a" containerID="6c78b38cd3de24c9f0a98751d052a0f786702ff3e44c7ec068a3476890934124" exitCode=0 Jul 29 14:42:58.846550 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:58.846291 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq" event={"ID":"4d060c49-d966-486c-bf4e-35b6c3573e0a","Type":"ContainerDied","Data":"6c78b38cd3de24c9f0a98751d052a0f786702ff3e44c7ec068a3476890934124"} Jul 29 14:42:58.848329 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:58.848310 2580 generic.go:356] "Generic (PLEG): container finished" podID="8c3f17c4-f6be-4645-b934-61a6a5499b6d" containerID="354e5eeac386f49772fc1824a9f37c3539e27dcb1130befe83e5e6089b3cc03c" exitCode=0 Jul 29 14:42:58.848420 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:58.848382 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s" event={"ID":"8c3f17c4-f6be-4645-b934-61a6a5499b6d","Type":"ContainerDied","Data":"354e5eeac386f49772fc1824a9f37c3539e27dcb1130befe83e5e6089b3cc03c"} Jul 29 14:42:58.850360 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:58.850343 2580 generic.go:356] "Generic (PLEG): container finished" podID="751fa8fe-b9a5-4026-bd77-f38f73957032" containerID="b07c70db801af9a4e85f6e51d4e7759e097d14d04f47d21c98f3fbc721a180a5" exitCode=0 Jul 29 14:42:58.850421 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:58.850400 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls" event={"ID":"751fa8fe-b9a5-4026-bd77-f38f73957032","Type":"ContainerDied","Data":"b07c70db801af9a4e85f6e51d4e7759e097d14d04f47d21c98f3fbc721a180a5"} Jul 29 14:42:58.851960 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:58.851942 2580 generic.go:356] "Generic (PLEG): container finished" podID="7936a6a4-72ef-49fe-b75d-48e925e0e807" containerID="4ce70814a7b69d7a57ac391df81e3f0455f5c8688b9cf1f7d31d8c0fcbe7602b" exitCode=0 Jul 29 14:42:58.852064 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:58.851996 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp" event={"ID":"7936a6a4-72ef-49fe-b75d-48e925e0e807","Type":"ContainerDied","Data":"4ce70814a7b69d7a57ac391df81e3f0455f5c8688b9cf1f7d31d8c0fcbe7602b"} Jul 29 14:42:59.862147 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:59.862114 2580 generic.go:356] "Generic (PLEG): container finished" podID="7936a6a4-72ef-49fe-b75d-48e925e0e807" containerID="c8d6d84e4b9d3c78cbedfe313a87c2e9f9c866b4418d28680bc9990f7da04b24" exitCode=0 Jul 29 14:42:59.862638 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:42:59.862185 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp" event={"ID":"7936a6a4-72ef-49fe-b75d-48e925e0e807","Type":"ContainerDied","Data":"c8d6d84e4b9d3c78cbedfe313a87c2e9f9c866b4418d28680bc9990f7da04b24"} Jul 29 14:43:00.018020 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.017996 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq" Jul 29 14:43:00.049831 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.049808 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s" Jul 29 14:43:00.052906 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.052885 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls" Jul 29 14:43:00.119057 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.119004 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c3f17c4-f6be-4645-b934-61a6a5499b6d-util\") pod \"8c3f17c4-f6be-4645-b934-61a6a5499b6d\" (UID: \"8c3f17c4-f6be-4645-b934-61a6a5499b6d\") " Jul 29 14:43:00.119057 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.119032 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/751fa8fe-b9a5-4026-bd77-f38f73957032-bundle\") pod \"751fa8fe-b9a5-4026-bd77-f38f73957032\" (UID: \"751fa8fe-b9a5-4026-bd77-f38f73957032\") " Jul 29 14:43:00.119188 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.119078 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q457b\" (UniqueName: \"kubernetes.io/projected/4d060c49-d966-486c-bf4e-35b6c3573e0a-kube-api-access-q457b\") pod \"4d060c49-d966-486c-bf4e-35b6c3573e0a\" (UID: \"4d060c49-d966-486c-bf4e-35b6c3573e0a\") " Jul 29 14:43:00.119188 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.119095 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d060c49-d966-486c-bf4e-35b6c3573e0a-util\") pod \"4d060c49-d966-486c-bf4e-35b6c3573e0a\" (UID: \"4d060c49-d966-486c-bf4e-35b6c3573e0a\") " Jul 29 14:43:00.119188 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.119126 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d060c49-d966-486c-bf4e-35b6c3573e0a-bundle\") pod \"4d060c49-d966-486c-bf4e-35b6c3573e0a\" (UID: \"4d060c49-d966-486c-bf4e-35b6c3573e0a\") " Jul 29 14:43:00.119188 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.119147 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxq6b\" (UniqueName: \"kubernetes.io/projected/8c3f17c4-f6be-4645-b934-61a6a5499b6d-kube-api-access-mxq6b\") pod \"8c3f17c4-f6be-4645-b934-61a6a5499b6d\" (UID: \"8c3f17c4-f6be-4645-b934-61a6a5499b6d\") " Jul 29 14:43:00.119188 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.119163 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/751fa8fe-b9a5-4026-bd77-f38f73957032-util\") pod \"751fa8fe-b9a5-4026-bd77-f38f73957032\" (UID: \"751fa8fe-b9a5-4026-bd77-f38f73957032\") " Jul 29 14:43:00.119188 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.119182 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgr7l\" (UniqueName: \"kubernetes.io/projected/751fa8fe-b9a5-4026-bd77-f38f73957032-kube-api-access-bgr7l\") pod \"751fa8fe-b9a5-4026-bd77-f38f73957032\" (UID: \"751fa8fe-b9a5-4026-bd77-f38f73957032\") " Jul 29 14:43:00.119469 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.119225 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c3f17c4-f6be-4645-b934-61a6a5499b6d-bundle\") pod \"8c3f17c4-f6be-4645-b934-61a6a5499b6d\" (UID: \"8c3f17c4-f6be-4645-b934-61a6a5499b6d\") " Jul 29 14:43:00.120143 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.120027 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d060c49-d966-486c-bf4e-35b6c3573e0a-bundle" (OuterVolumeSpecName: "bundle") pod "4d060c49-d966-486c-bf4e-35b6c3573e0a" (UID: "4d060c49-d966-486c-bf4e-35b6c3573e0a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:43:00.120143 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.120071 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/751fa8fe-b9a5-4026-bd77-f38f73957032-bundle" (OuterVolumeSpecName: "bundle") pod "751fa8fe-b9a5-4026-bd77-f38f73957032" (UID: "751fa8fe-b9a5-4026-bd77-f38f73957032"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:43:00.120143 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.120131 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c3f17c4-f6be-4645-b934-61a6a5499b6d-bundle" (OuterVolumeSpecName: "bundle") pod "8c3f17c4-f6be-4645-b934-61a6a5499b6d" (UID: "8c3f17c4-f6be-4645-b934-61a6a5499b6d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:43:00.121870 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.121830 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c3f17c4-f6be-4645-b934-61a6a5499b6d-kube-api-access-mxq6b" (OuterVolumeSpecName: "kube-api-access-mxq6b") pod "8c3f17c4-f6be-4645-b934-61a6a5499b6d" (UID: "8c3f17c4-f6be-4645-b934-61a6a5499b6d"). InnerVolumeSpecName "kube-api-access-mxq6b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 14:43:00.121979 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.121959 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d060c49-d966-486c-bf4e-35b6c3573e0a-kube-api-access-q457b" (OuterVolumeSpecName: "kube-api-access-q457b") pod "4d060c49-d966-486c-bf4e-35b6c3573e0a" (UID: "4d060c49-d966-486c-bf4e-35b6c3573e0a"). InnerVolumeSpecName "kube-api-access-q457b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 14:43:00.122194 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.122164 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/751fa8fe-b9a5-4026-bd77-f38f73957032-kube-api-access-bgr7l" (OuterVolumeSpecName: "kube-api-access-bgr7l") pod "751fa8fe-b9a5-4026-bd77-f38f73957032" (UID: "751fa8fe-b9a5-4026-bd77-f38f73957032"). InnerVolumeSpecName "kube-api-access-bgr7l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 14:43:00.125504 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.125476 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/751fa8fe-b9a5-4026-bd77-f38f73957032-util" (OuterVolumeSpecName: "util") pod "751fa8fe-b9a5-4026-bd77-f38f73957032" (UID: "751fa8fe-b9a5-4026-bd77-f38f73957032"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:43:00.125750 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.125716 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d060c49-d966-486c-bf4e-35b6c3573e0a-util" (OuterVolumeSpecName: "util") pod "4d060c49-d966-486c-bf4e-35b6c3573e0a" (UID: "4d060c49-d966-486c-bf4e-35b6c3573e0a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:43:00.127321 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.127299 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c3f17c4-f6be-4645-b934-61a6a5499b6d-util" (OuterVolumeSpecName: "util") pod "8c3f17c4-f6be-4645-b934-61a6a5499b6d" (UID: "8c3f17c4-f6be-4645-b934-61a6a5499b6d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:43:00.219775 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.219753 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8c3f17c4-f6be-4645-b934-61a6a5499b6d-util\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:00.219775 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.219771 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/751fa8fe-b9a5-4026-bd77-f38f73957032-bundle\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:00.219891 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.219779 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q457b\" (UniqueName: \"kubernetes.io/projected/4d060c49-d966-486c-bf4e-35b6c3573e0a-kube-api-access-q457b\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:00.219891 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.219787 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4d060c49-d966-486c-bf4e-35b6c3573e0a-util\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:00.219891 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.219795 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4d060c49-d966-486c-bf4e-35b6c3573e0a-bundle\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:00.219891 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.219802 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxq6b\" (UniqueName: \"kubernetes.io/projected/8c3f17c4-f6be-4645-b934-61a6a5499b6d-kube-api-access-mxq6b\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:00.219891 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.219811 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/751fa8fe-b9a5-4026-bd77-f38f73957032-util\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:00.219891 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.219818 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgr7l\" (UniqueName: \"kubernetes.io/projected/751fa8fe-b9a5-4026-bd77-f38f73957032-kube-api-access-bgr7l\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:00.219891 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.219826 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8c3f17c4-f6be-4645-b934-61a6a5499b6d-bundle\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:00.869562 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.869532 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq" event={"ID":"4d060c49-d966-486c-bf4e-35b6c3573e0a","Type":"ContainerDied","Data":"cc27f697be4090f314669c68c1414cff004ccc5edd138c3f614ce33172549add"} Jul 29 14:43:00.869960 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.869572 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc27f697be4090f314669c68c1414cff004ccc5edd138c3f614ce33172549add" Jul 29 14:43:00.869960 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.869536 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq" Jul 29 14:43:00.871906 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.871872 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s" Jul 29 14:43:00.872033 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.871900 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s" event={"ID":"8c3f17c4-f6be-4645-b934-61a6a5499b6d","Type":"ContainerDied","Data":"0e1ee9e1a13dedb926a66ea7863589ac5ebc94f3ae73555e161b3b201d148310"} Jul 29 14:43:00.872033 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.871946 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e1ee9e1a13dedb926a66ea7863589ac5ebc94f3ae73555e161b3b201d148310" Jul 29 14:43:00.874270 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.874248 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls" Jul 29 14:43:00.874382 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.874278 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls" event={"ID":"751fa8fe-b9a5-4026-bd77-f38f73957032","Type":"ContainerDied","Data":"7985bddecc0058ea1e1283741e95324ed44924942f14a629ff0f7d11c0127f1a"} Jul 29 14:43:00.874382 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:00.874329 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7985bddecc0058ea1e1283741e95324ed44924942f14a629ff0f7d11c0127f1a" Jul 29 14:43:01.008135 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:01.008115 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp" Jul 29 14:43:01.126523 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:01.126463 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gx9km\" (UniqueName: \"kubernetes.io/projected/7936a6a4-72ef-49fe-b75d-48e925e0e807-kube-api-access-gx9km\") pod \"7936a6a4-72ef-49fe-b75d-48e925e0e807\" (UID: \"7936a6a4-72ef-49fe-b75d-48e925e0e807\") " Jul 29 14:43:01.126523 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:01.126503 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7936a6a4-72ef-49fe-b75d-48e925e0e807-util\") pod \"7936a6a4-72ef-49fe-b75d-48e925e0e807\" (UID: \"7936a6a4-72ef-49fe-b75d-48e925e0e807\") " Jul 29 14:43:01.126694 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:01.126542 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7936a6a4-72ef-49fe-b75d-48e925e0e807-bundle\") pod \"7936a6a4-72ef-49fe-b75d-48e925e0e807\" (UID: \"7936a6a4-72ef-49fe-b75d-48e925e0e807\") " Jul 29 14:43:01.127127 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:01.127101 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7936a6a4-72ef-49fe-b75d-48e925e0e807-bundle" (OuterVolumeSpecName: "bundle") pod "7936a6a4-72ef-49fe-b75d-48e925e0e807" (UID: "7936a6a4-72ef-49fe-b75d-48e925e0e807"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:43:01.128475 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:01.128448 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7936a6a4-72ef-49fe-b75d-48e925e0e807-kube-api-access-gx9km" (OuterVolumeSpecName: "kube-api-access-gx9km") pod "7936a6a4-72ef-49fe-b75d-48e925e0e807" (UID: "7936a6a4-72ef-49fe-b75d-48e925e0e807"). InnerVolumeSpecName "kube-api-access-gx9km". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 14:43:01.131621 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:01.131591 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7936a6a4-72ef-49fe-b75d-48e925e0e807-util" (OuterVolumeSpecName: "util") pod "7936a6a4-72ef-49fe-b75d-48e925e0e807" (UID: "7936a6a4-72ef-49fe-b75d-48e925e0e807"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:43:01.228121 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:01.228090 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gx9km\" (UniqueName: \"kubernetes.io/projected/7936a6a4-72ef-49fe-b75d-48e925e0e807-kube-api-access-gx9km\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:01.228121 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:01.228120 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7936a6a4-72ef-49fe-b75d-48e925e0e807-util\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:01.228121 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:01.228130 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7936a6a4-72ef-49fe-b75d-48e925e0e807-bundle\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:01.881175 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:01.881152 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp" Jul 29 14:43:01.881464 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:01.881168 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp" event={"ID":"7936a6a4-72ef-49fe-b75d-48e925e0e807","Type":"ContainerDied","Data":"103149f373d78e605c63daaae5547335d8cd9ea1f8796777f32cffac661e4fa2"} Jul 29 14:43:01.881464 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:01.881211 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="103149f373d78e605c63daaae5547335d8cd9ea1f8796777f32cffac661e4fa2" Jul 29 14:43:02.773326 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:02.773290 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:05.250370 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.250331 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84f665d64b-vrrcr"] Jul 29 14:43:05.250894 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.250863 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c3f17c4-f6be-4645-b934-61a6a5499b6d" containerName="pull" Jul 29 14:43:05.250894 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.250884 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c3f17c4-f6be-4645-b934-61a6a5499b6d" containerName="pull" Jul 29 14:43:05.251113 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.250899 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c3f17c4-f6be-4645-b934-61a6a5499b6d" containerName="util" Jul 29 14:43:05.251113 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.250908 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c3f17c4-f6be-4645-b934-61a6a5499b6d" containerName="util" Jul 29 14:43:05.251113 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.250921 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="751fa8fe-b9a5-4026-bd77-f38f73957032" containerName="util" Jul 29 14:43:05.251113 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.250931 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="751fa8fe-b9a5-4026-bd77-f38f73957032" containerName="util" Jul 29 14:43:05.251113 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.250948 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d060c49-d966-486c-bf4e-35b6c3573e0a" containerName="util" Jul 29 14:43:05.251113 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.251006 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d060c49-d966-486c-bf4e-35b6c3573e0a" containerName="util" Jul 29 14:43:05.251113 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.251020 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7936a6a4-72ef-49fe-b75d-48e925e0e807" containerName="pull" Jul 29 14:43:05.251113 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.251030 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7936a6a4-72ef-49fe-b75d-48e925e0e807" containerName="pull" Jul 29 14:43:05.251113 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.251040 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7936a6a4-72ef-49fe-b75d-48e925e0e807" containerName="extract" Jul 29 14:43:05.251113 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.251048 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7936a6a4-72ef-49fe-b75d-48e925e0e807" containerName="extract" Jul 29 14:43:05.251113 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.251068 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c3f17c4-f6be-4645-b934-61a6a5499b6d" containerName="extract" Jul 29 14:43:05.251113 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.251077 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c3f17c4-f6be-4645-b934-61a6a5499b6d" containerName="extract" Jul 29 14:43:05.251113 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.251085 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d060c49-d966-486c-bf4e-35b6c3573e0a" containerName="extract" Jul 29 14:43:05.251113 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.251094 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d060c49-d966-486c-bf4e-35b6c3573e0a" containerName="extract" Jul 29 14:43:05.251113 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.251104 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7936a6a4-72ef-49fe-b75d-48e925e0e807" containerName="util" Jul 29 14:43:05.251113 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.251111 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7936a6a4-72ef-49fe-b75d-48e925e0e807" containerName="util" Jul 29 14:43:05.251874 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.251133 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="751fa8fe-b9a5-4026-bd77-f38f73957032" containerName="extract" Jul 29 14:43:05.251874 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.251141 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="751fa8fe-b9a5-4026-bd77-f38f73957032" containerName="extract" Jul 29 14:43:05.251874 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.251159 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d060c49-d966-486c-bf4e-35b6c3573e0a" containerName="pull" Jul 29 14:43:05.251874 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.251169 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d060c49-d966-486c-bf4e-35b6c3573e0a" containerName="pull" Jul 29 14:43:05.251874 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.251178 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="751fa8fe-b9a5-4026-bd77-f38f73957032" containerName="pull" Jul 29 14:43:05.251874 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.251187 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="751fa8fe-b9a5-4026-bd77-f38f73957032" containerName="pull" Jul 29 14:43:05.251874 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.251296 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="751fa8fe-b9a5-4026-bd77-f38f73957032" containerName="extract" Jul 29 14:43:05.251874 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.251342 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c3f17c4-f6be-4645-b934-61a6a5499b6d" containerName="extract" Jul 29 14:43:05.251874 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.251355 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d060c49-d966-486c-bf4e-35b6c3573e0a" containerName="extract" Jul 29 14:43:05.251874 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.251365 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7936a6a4-72ef-49fe-b75d-48e925e0e807" containerName="extract" Jul 29 14:43:05.254905 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.254881 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.265908 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.265883 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84f665d64b-vrrcr"] Jul 29 14:43:05.308011 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.307975 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:05.356108 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.356079 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa91c527-597c-4d17-b213-d87c5273cfc0-console-serving-cert\") pod \"console-84f665d64b-vrrcr\" (UID: \"aa91c527-597c-4d17-b213-d87c5273cfc0\") " pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.356232 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.356110 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa91c527-597c-4d17-b213-d87c5273cfc0-oauth-serving-cert\") pod \"console-84f665d64b-vrrcr\" (UID: \"aa91c527-597c-4d17-b213-d87c5273cfc0\") " pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.356232 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.356141 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bl6k\" (UniqueName: \"kubernetes.io/projected/aa91c527-597c-4d17-b213-d87c5273cfc0-kube-api-access-6bl6k\") pod \"console-84f665d64b-vrrcr\" (UID: \"aa91c527-597c-4d17-b213-d87c5273cfc0\") " pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.356331 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.356224 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa91c527-597c-4d17-b213-d87c5273cfc0-service-ca\") pod \"console-84f665d64b-vrrcr\" (UID: \"aa91c527-597c-4d17-b213-d87c5273cfc0\") " pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.356331 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.356266 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa91c527-597c-4d17-b213-d87c5273cfc0-console-config\") pod \"console-84f665d64b-vrrcr\" (UID: \"aa91c527-597c-4d17-b213-d87c5273cfc0\") " pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.356331 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.356321 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa91c527-597c-4d17-b213-d87c5273cfc0-console-oauth-config\") pod \"console-84f665d64b-vrrcr\" (UID: \"aa91c527-597c-4d17-b213-d87c5273cfc0\") " pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.356423 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.356341 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa91c527-597c-4d17-b213-d87c5273cfc0-trusted-ca-bundle\") pod \"console-84f665d64b-vrrcr\" (UID: \"aa91c527-597c-4d17-b213-d87c5273cfc0\") " pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.378434 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.378395 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:05.457467 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.457440 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bl6k\" (UniqueName: \"kubernetes.io/projected/aa91c527-597c-4d17-b213-d87c5273cfc0-kube-api-access-6bl6k\") pod \"console-84f665d64b-vrrcr\" (UID: \"aa91c527-597c-4d17-b213-d87c5273cfc0\") " pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.457694 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.457475 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa91c527-597c-4d17-b213-d87c5273cfc0-service-ca\") pod \"console-84f665d64b-vrrcr\" (UID: \"aa91c527-597c-4d17-b213-d87c5273cfc0\") " pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.457694 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.457496 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa91c527-597c-4d17-b213-d87c5273cfc0-console-config\") pod \"console-84f665d64b-vrrcr\" (UID: \"aa91c527-597c-4d17-b213-d87c5273cfc0\") " pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.457694 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.457517 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa91c527-597c-4d17-b213-d87c5273cfc0-console-oauth-config\") pod \"console-84f665d64b-vrrcr\" (UID: \"aa91c527-597c-4d17-b213-d87c5273cfc0\") " pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.457694 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.457565 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa91c527-597c-4d17-b213-d87c5273cfc0-trusted-ca-bundle\") pod \"console-84f665d64b-vrrcr\" (UID: \"aa91c527-597c-4d17-b213-d87c5273cfc0\") " pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.457694 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.457604 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa91c527-597c-4d17-b213-d87c5273cfc0-console-serving-cert\") pod \"console-84f665d64b-vrrcr\" (UID: \"aa91c527-597c-4d17-b213-d87c5273cfc0\") " pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.457694 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.457618 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa91c527-597c-4d17-b213-d87c5273cfc0-oauth-serving-cert\") pod \"console-84f665d64b-vrrcr\" (UID: \"aa91c527-597c-4d17-b213-d87c5273cfc0\") " pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.458367 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.458339 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/aa91c527-597c-4d17-b213-d87c5273cfc0-console-config\") pod \"console-84f665d64b-vrrcr\" (UID: \"aa91c527-597c-4d17-b213-d87c5273cfc0\") " pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.458475 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.458404 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/aa91c527-597c-4d17-b213-d87c5273cfc0-service-ca\") pod \"console-84f665d64b-vrrcr\" (UID: \"aa91c527-597c-4d17-b213-d87c5273cfc0\") " pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.458536 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.458491 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa91c527-597c-4d17-b213-d87c5273cfc0-trusted-ca-bundle\") pod \"console-84f665d64b-vrrcr\" (UID: \"aa91c527-597c-4d17-b213-d87c5273cfc0\") " pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.458536 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.458488 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/aa91c527-597c-4d17-b213-d87c5273cfc0-oauth-serving-cert\") pod \"console-84f665d64b-vrrcr\" (UID: \"aa91c527-597c-4d17-b213-d87c5273cfc0\") " pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.460235 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.460212 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/aa91c527-597c-4d17-b213-d87c5273cfc0-console-serving-cert\") pod \"console-84f665d64b-vrrcr\" (UID: \"aa91c527-597c-4d17-b213-d87c5273cfc0\") " pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.460336 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.460263 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/aa91c527-597c-4d17-b213-d87c5273cfc0-console-oauth-config\") pod \"console-84f665d64b-vrrcr\" (UID: \"aa91c527-597c-4d17-b213-d87c5273cfc0\") " pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.465440 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.465418 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bl6k\" (UniqueName: \"kubernetes.io/projected/aa91c527-597c-4d17-b213-d87c5273cfc0-kube-api-access-6bl6k\") pod \"console-84f665d64b-vrrcr\" (UID: \"aa91c527-597c-4d17-b213-d87c5273cfc0\") " pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.564690 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.564618 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:05.697179 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.697155 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84f665d64b-vrrcr"] Jul 29 14:43:05.698819 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:43:05.698792 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa91c527_597c_4d17_b213_d87c5273cfc0.slice/crio-3453db79ee4872e352ce3a47dd113dd826ae52465a4416fd5a33d064ca89d17d WatchSource:0}: Error finding container 3453db79ee4872e352ce3a47dd113dd826ae52465a4416fd5a33d064ca89d17d: Status 404 returned error can't find the container with id 3453db79ee4872e352ce3a47dd113dd826ae52465a4416fd5a33d064ca89d17d Jul 29 14:43:05.908486 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.908396 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84f665d64b-vrrcr" event={"ID":"aa91c527-597c-4d17-b213-d87c5273cfc0","Type":"ContainerStarted","Data":"5529863146ebb02540c09aac08103c3faa1e5a13013df042bd4af12a6a6533cc"} Jul 29 14:43:05.908486 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.908451 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84f665d64b-vrrcr" event={"ID":"aa91c527-597c-4d17-b213-d87c5273cfc0","Type":"ContainerStarted","Data":"3453db79ee4872e352ce3a47dd113dd826ae52465a4416fd5a33d064ca89d17d"} Jul 29 14:43:05.928072 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:05.928025 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84f665d64b-vrrcr" podStartSLOduration=0.928011297 podStartE2EDuration="928.011297ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:43:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:43:05.926057557 +0000 UTC m=+462.654206748" watchObservedRunningTime="2026-07-29 14:43:05.928011297 +0000 UTC m=+462.656160491" Jul 29 14:43:07.826803 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:07.826763 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:08.448019 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:08.447982 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:11.890829 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:11.890788 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-rjtbg"] Jul 29 14:43:11.894783 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:11.894759 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rjtbg" Jul 29 14:43:11.897497 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:11.897469 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 29 14:43:11.898154 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:11.898129 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-8ksdx\"" Jul 29 14:43:11.898273 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:11.898187 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 29 14:43:11.911630 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:11.911606 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-rjtbg"] Jul 29 14:43:12.010070 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:12.010046 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjw8k\" (UniqueName: \"kubernetes.io/projected/d61df5ab-7566-485f-9ea1-4e181286254a-kube-api-access-vjw8k\") pod \"authorino-operator-75cc8d5cb7-rjtbg\" (UID: \"d61df5ab-7566-485f-9ea1-4e181286254a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-rjtbg" Jul 29 14:43:12.110976 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:12.110942 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjw8k\" (UniqueName: \"kubernetes.io/projected/d61df5ab-7566-485f-9ea1-4e181286254a-kube-api-access-vjw8k\") pod \"authorino-operator-75cc8d5cb7-rjtbg\" (UID: \"d61df5ab-7566-485f-9ea1-4e181286254a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-rjtbg" Jul 29 14:43:12.125003 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:12.124981 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjw8k\" (UniqueName: \"kubernetes.io/projected/d61df5ab-7566-485f-9ea1-4e181286254a-kube-api-access-vjw8k\") pod \"authorino-operator-75cc8d5cb7-rjtbg\" (UID: \"d61df5ab-7566-485f-9ea1-4e181286254a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-rjtbg" Jul 29 14:43:12.206876 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:12.206810 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rjtbg" Jul 29 14:43:12.342223 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:12.342197 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-rjtbg"] Jul 29 14:43:12.344173 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:43:12.344145 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd61df5ab_7566_485f_9ea1_4e181286254a.slice/crio-4df22dc48bb7fb4191b1a0fa8f95d68d08ffc67284ff4175dd0f1dd3091a7345 WatchSource:0}: Error finding container 4df22dc48bb7fb4191b1a0fa8f95d68d08ffc67284ff4175dd0f1dd3091a7345: Status 404 returned error can't find the container with id 4df22dc48bb7fb4191b1a0fa8f95d68d08ffc67284ff4175dd0f1dd3091a7345 Jul 29 14:43:12.947193 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:12.947160 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rjtbg" event={"ID":"d61df5ab-7566-485f-9ea1-4e181286254a","Type":"ContainerStarted","Data":"4df22dc48bb7fb4191b1a0fa8f95d68d08ffc67284ff4175dd0f1dd3091a7345"} Jul 29 14:43:13.241236 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:13.241104 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:14.965581 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:14.965548 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rjtbg" event={"ID":"d61df5ab-7566-485f-9ea1-4e181286254a","Type":"ContainerStarted","Data":"db1890004965fe3c015cf1c43fb5cc203af676554312e59d293dbedc9a33ca33"} Jul 29 14:43:14.965932 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:14.965620 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rjtbg" Jul 29 14:43:14.983788 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:14.983744 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rjtbg" podStartSLOduration=1.9693180510000001 podStartE2EDuration="3.98371414s" totalImagesPullingTime="2.014396089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:43:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:43:14.981936941 +0000 UTC m=+471.710086131" watchObservedRunningTime="2026-07-29 14:43:14.98371414 +0000 UTC m=+471.711863395" Jul 29 14:43:15.332201 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:15.332174 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:15.547653 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:15.547623 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:15.564842 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:15.564823 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:15.564956 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:15.564877 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:15.569133 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:15.569115 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:15.838096 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:15.838074 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:15.973195 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:15.973175 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84f665d64b-vrrcr" Jul 29 14:43:16.033372 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:16.033348 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69467bccf7-cpbq6"] Jul 29 14:43:16.177903 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:16.177825 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:16.273868 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:16.273844 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:16.372890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:16.372851 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:16.486439 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:16.486388 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:17.696484 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:17.696453 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:19.470956 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:19.470917 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:20.138524 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:20.138493 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45"] Jul 29 14:43:20.142044 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:20.142025 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" Jul 29 14:43:20.144251 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:20.144220 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rhxm9\"" Jul 29 14:43:20.155011 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:20.154992 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45"] Jul 29 14:43:20.277841 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:20.277809 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1aab703d-a990-4166-a5ae-97b7f3eac0e1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lrv45\" (UID: \"1aab703d-a990-4166-a5ae-97b7f3eac0e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" Jul 29 14:43:20.277964 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:20.277847 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjpsh\" (UniqueName: \"kubernetes.io/projected/1aab703d-a990-4166-a5ae-97b7f3eac0e1-kube-api-access-fjpsh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lrv45\" (UID: \"1aab703d-a990-4166-a5ae-97b7f3eac0e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" Jul 29 14:43:20.379226 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:20.379195 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1aab703d-a990-4166-a5ae-97b7f3eac0e1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lrv45\" (UID: \"1aab703d-a990-4166-a5ae-97b7f3eac0e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" Jul 29 14:43:20.379315 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:20.379230 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjpsh\" (UniqueName: \"kubernetes.io/projected/1aab703d-a990-4166-a5ae-97b7f3eac0e1-kube-api-access-fjpsh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lrv45\" (UID: \"1aab703d-a990-4166-a5ae-97b7f3eac0e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" Jul 29 14:43:20.379574 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:20.379554 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1aab703d-a990-4166-a5ae-97b7f3eac0e1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lrv45\" (UID: \"1aab703d-a990-4166-a5ae-97b7f3eac0e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" Jul 29 14:43:20.389526 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:20.389466 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjpsh\" (UniqueName: \"kubernetes.io/projected/1aab703d-a990-4166-a5ae-97b7f3eac0e1-kube-api-access-fjpsh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lrv45\" (UID: \"1aab703d-a990-4166-a5ae-97b7f3eac0e1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" Jul 29 14:43:20.452449 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:20.452424 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" Jul 29 14:43:20.599456 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:20.599310 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45"] Jul 29 14:43:20.602313 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:43:20.602269 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aab703d_a990_4166_a5ae_97b7f3eac0e1.slice/crio-55fabff0f61718739683a500ba1af67bd5295388fafa7ed1d0011f6e9f536214 WatchSource:0}: Error finding container 55fabff0f61718739683a500ba1af67bd5295388fafa7ed1d0011f6e9f536214: Status 404 returned error can't find the container with id 55fabff0f61718739683a500ba1af67bd5295388fafa7ed1d0011f6e9f536214 Jul 29 14:43:20.997069 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:20.997039 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" event={"ID":"1aab703d-a990-4166-a5ae-97b7f3eac0e1","Type":"ContainerStarted","Data":"55fabff0f61718739683a500ba1af67bd5295388fafa7ed1d0011f6e9f536214"} Jul 29 14:43:21.058944 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:21.058915 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:22.382448 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:22.382397 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:24.498271 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:24.498233 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:24.575934 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:24.575899 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:24.681142 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:24.681101 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:24.782661 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:24.782576 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:25.972621 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:25.972539 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rjtbg" Jul 29 14:43:26.026328 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:26.026292 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" event={"ID":"1aab703d-a990-4166-a5ae-97b7f3eac0e1","Type":"ContainerStarted","Data":"76d62369b4d86f456f229a55153da797ec513b7db09b975b1b4852fd6f1e9da8"} Jul 29 14:43:26.026474 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:26.026358 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" Jul 29 14:43:26.051055 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:26.051013 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" podStartSLOduration=1.103192686 podStartE2EDuration="6.050998609s" totalImagesPullingTime="4.947805923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:43:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:43:26.050404291 +0000 UTC m=+482.778553492" watchObservedRunningTime="2026-07-29 14:43:26.050998609 +0000 UTC m=+482.779147796" Jul 29 14:43:28.156293 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:28.156259 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:28.251784 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:28.251753 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:28.352113 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:28.352091 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:31.457234 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:31.457199 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:31.556737 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:31.556698 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:36.160766 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:36.160710 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:36.261845 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:36.261809 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:37.034516 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:37.034488 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" Jul 29 14:43:37.978694 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:37.978658 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:38.080524 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.080491 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:38.186252 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.186211 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh"] Jul 29 14:43:38.192823 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.192801 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh" Jul 29 14:43:38.200138 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.200113 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh"] Jul 29 14:43:38.220222 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.220199 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwvx4\" (UniqueName: \"kubernetes.io/projected/15cd006e-41be-4387-b3cc-846de47f2e24-kube-api-access-bwvx4\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-ntvrh\" (UID: \"15cd006e-41be-4387-b3cc-846de47f2e24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh" Jul 29 14:43:38.220346 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.220261 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15cd006e-41be-4387-b3cc-846de47f2e24-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-ntvrh\" (UID: \"15cd006e-41be-4387-b3cc-846de47f2e24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh" Jul 29 14:43:38.296105 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.296078 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh"] Jul 29 14:43:38.296268 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:43:38.296251 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-bwvx4], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh" podUID="15cd006e-41be-4387-b3cc-846de47f2e24" Jul 29 14:43:38.320738 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.320704 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15cd006e-41be-4387-b3cc-846de47f2e24-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-ntvrh\" (UID: \"15cd006e-41be-4387-b3cc-846de47f2e24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh" Jul 29 14:43:38.320864 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.320851 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwvx4\" (UniqueName: \"kubernetes.io/projected/15cd006e-41be-4387-b3cc-846de47f2e24-kube-api-access-bwvx4\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-ntvrh\" (UID: \"15cd006e-41be-4387-b3cc-846de47f2e24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh" Jul 29 14:43:38.321079 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.321060 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15cd006e-41be-4387-b3cc-846de47f2e24-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-ntvrh\" (UID: \"15cd006e-41be-4387-b3cc-846de47f2e24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh" Jul 29 14:43:38.329715 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.329693 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lllw7"] Jul 29 14:43:38.331331 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.331309 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwvx4\" (UniqueName: \"kubernetes.io/projected/15cd006e-41be-4387-b3cc-846de47f2e24-kube-api-access-bwvx4\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-ntvrh\" (UID: \"15cd006e-41be-4387-b3cc-846de47f2e24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh" Jul 29 14:43:38.333038 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.333022 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lllw7" Jul 29 14:43:38.348371 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.348348 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lllw7"] Jul 29 14:43:38.421516 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.421495 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ggqz\" (UniqueName: \"kubernetes.io/projected/7e74dacb-497f-4f6a-9c08-22c0fa028cd1-kube-api-access-7ggqz\") pod \"kuadrant-operator-controller-manager-55467d9f7-lllw7\" (UID: \"7e74dacb-497f-4f6a-9c08-22c0fa028cd1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lllw7" Jul 29 14:43:38.421608 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.421531 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e74dacb-497f-4f6a-9c08-22c0fa028cd1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-lllw7\" (UID: \"7e74dacb-497f-4f6a-9c08-22c0fa028cd1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lllw7" Jul 29 14:43:38.521942 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.521910 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e74dacb-497f-4f6a-9c08-22c0fa028cd1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-lllw7\" (UID: \"7e74dacb-497f-4f6a-9c08-22c0fa028cd1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lllw7" Jul 29 14:43:38.522045 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.521997 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ggqz\" (UniqueName: \"kubernetes.io/projected/7e74dacb-497f-4f6a-9c08-22c0fa028cd1-kube-api-access-7ggqz\") pod \"kuadrant-operator-controller-manager-55467d9f7-lllw7\" (UID: \"7e74dacb-497f-4f6a-9c08-22c0fa028cd1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lllw7" Jul 29 14:43:38.522259 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.522241 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e74dacb-497f-4f6a-9c08-22c0fa028cd1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-lllw7\" (UID: \"7e74dacb-497f-4f6a-9c08-22c0fa028cd1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lllw7" Jul 29 14:43:38.529550 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.529528 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ggqz\" (UniqueName: \"kubernetes.io/projected/7e74dacb-497f-4f6a-9c08-22c0fa028cd1-kube-api-access-7ggqz\") pod \"kuadrant-operator-controller-manager-55467d9f7-lllw7\" (UID: \"7e74dacb-497f-4f6a-9c08-22c0fa028cd1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lllw7" Jul 29 14:43:38.647402 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.647352 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lllw7" Jul 29 14:43:38.794255 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.794223 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lllw7"] Jul 29 14:43:38.797428 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:43:38.797400 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e74dacb_497f_4f6a_9c08_22c0fa028cd1.slice/crio-d4e34c4ad6ed86b24df6842b6b7281d542006ac01200672fc0a817f2bbdf57f8 WatchSource:0}: Error finding container d4e34c4ad6ed86b24df6842b6b7281d542006ac01200672fc0a817f2bbdf57f8: Status 404 returned error can't find the container with id d4e34c4ad6ed86b24df6842b6b7281d542006ac01200672fc0a817f2bbdf57f8 Jul 29 14:43:38.812857 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.812835 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lllw7"] Jul 29 14:43:38.825492 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.825471 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lllw7"] Jul 29 14:43:38.835162 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.835142 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh"] Jul 29 14:43:38.841500 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.841425 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc"] Jul 29 14:43:38.847314 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.847294 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh"] Jul 29 14:43:38.847415 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.847377 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc" Jul 29 14:43:38.859183 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.859163 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc"] Jul 29 14:43:38.862366 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.862345 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45"] Jul 29 14:43:38.862564 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.862539 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" podUID="1aab703d-a990-4166-a5ae-97b7f3eac0e1" containerName="manager" containerID="cri-o://76d62369b4d86f456f229a55153da797ec513b7db09b975b1b4852fd6f1e9da8" gracePeriod=2 Jul 29 14:43:38.875585 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.875561 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45"] Jul 29 14:43:38.880474 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:43:38.880439 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lrv45\" is forbidden: User \"system:node:ip-10-0-132-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-42.ec2.internal' and this object" podUID="1aab703d-a990-4166-a5ae-97b7f3eac0e1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" Jul 29 14:43:38.894900 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.894881 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk"] Jul 29 14:43:38.895205 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.895193 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e74dacb-497f-4f6a-9c08-22c0fa028cd1" containerName="manager" Jul 29 14:43:38.895254 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.895206 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e74dacb-497f-4f6a-9c08-22c0fa028cd1" containerName="manager" Jul 29 14:43:38.895254 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.895224 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1aab703d-a990-4166-a5ae-97b7f3eac0e1" containerName="manager" Jul 29 14:43:38.895254 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.895231 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1aab703d-a990-4166-a5ae-97b7f3eac0e1" containerName="manager" Jul 29 14:43:38.895357 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.895278 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1aab703d-a990-4166-a5ae-97b7f3eac0e1" containerName="manager" Jul 29 14:43:38.895357 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.895287 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e74dacb-497f-4f6a-9c08-22c0fa028cd1" containerName="manager" Jul 29 14:43:38.898132 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.898091 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk" Jul 29 14:43:38.910808 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.910787 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk"] Jul 29 14:43:38.924599 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.924569 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08d58c35-37e9-4d28-9b7f-aa44abd525b3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x7nzk\" (UID: \"08d58c35-37e9-4d28-9b7f-aa44abd525b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk" Jul 29 14:43:38.924709 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.924611 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ebd67da2-8a88-43d6-9e62-b1d67553fae1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-w86wc\" (UID: \"ebd67da2-8a88-43d6-9e62-b1d67553fae1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc" Jul 29 14:43:38.924709 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.924653 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js8b7\" (UniqueName: \"kubernetes.io/projected/08d58c35-37e9-4d28-9b7f-aa44abd525b3-kube-api-access-js8b7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x7nzk\" (UID: \"08d58c35-37e9-4d28-9b7f-aa44abd525b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk" Jul 29 14:43:38.924845 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:38.924752 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8drt\" (UniqueName: \"kubernetes.io/projected/ebd67da2-8a88-43d6-9e62-b1d67553fae1-kube-api-access-p8drt\") pod \"kuadrant-operator-controller-manager-55467d9f7-w86wc\" (UID: \"ebd67da2-8a88-43d6-9e62-b1d67553fae1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc" Jul 29 14:43:38.925399 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:43:38.925371 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lrv45\" is forbidden: User \"system:node:ip-10-0-132-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-42.ec2.internal' and this object" podUID="1aab703d-a990-4166-a5ae-97b7f3eac0e1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" Jul 29 14:43:39.025143 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.025110 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08d58c35-37e9-4d28-9b7f-aa44abd525b3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x7nzk\" (UID: \"08d58c35-37e9-4d28-9b7f-aa44abd525b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk" Jul 29 14:43:39.025557 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.025149 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ebd67da2-8a88-43d6-9e62-b1d67553fae1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-w86wc\" (UID: \"ebd67da2-8a88-43d6-9e62-b1d67553fae1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc" Jul 29 14:43:39.025557 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.025178 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js8b7\" (UniqueName: \"kubernetes.io/projected/08d58c35-37e9-4d28-9b7f-aa44abd525b3-kube-api-access-js8b7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x7nzk\" (UID: \"08d58c35-37e9-4d28-9b7f-aa44abd525b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk" Jul 29 14:43:39.025557 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.025224 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8drt\" (UniqueName: \"kubernetes.io/projected/ebd67da2-8a88-43d6-9e62-b1d67553fae1-kube-api-access-p8drt\") pod \"kuadrant-operator-controller-manager-55467d9f7-w86wc\" (UID: \"ebd67da2-8a88-43d6-9e62-b1d67553fae1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc" Jul 29 14:43:39.025557 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.025509 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ebd67da2-8a88-43d6-9e62-b1d67553fae1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-w86wc\" (UID: \"ebd67da2-8a88-43d6-9e62-b1d67553fae1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc" Jul 29 14:43:39.025557 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.025527 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08d58c35-37e9-4d28-9b7f-aa44abd525b3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x7nzk\" (UID: \"08d58c35-37e9-4d28-9b7f-aa44abd525b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk" Jul 29 14:43:39.036445 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.036195 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8drt\" (UniqueName: \"kubernetes.io/projected/ebd67da2-8a88-43d6-9e62-b1d67553fae1-kube-api-access-p8drt\") pod \"kuadrant-operator-controller-manager-55467d9f7-w86wc\" (UID: \"ebd67da2-8a88-43d6-9e62-b1d67553fae1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc" Jul 29 14:43:39.036445 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.036234 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-js8b7\" (UniqueName: \"kubernetes.io/projected/08d58c35-37e9-4d28-9b7f-aa44abd525b3-kube-api-access-js8b7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-x7nzk\" (UID: \"08d58c35-37e9-4d28-9b7f-aa44abd525b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk" Jul 29 14:43:39.096178 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.096155 2580 generic.go:356] "Generic (PLEG): container finished" podID="1aab703d-a990-4166-a5ae-97b7f3eac0e1" containerID="76d62369b4d86f456f229a55153da797ec513b7db09b975b1b4852fd6f1e9da8" exitCode=0 Jul 29 14:43:39.096289 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.096238 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55fabff0f61718739683a500ba1af67bd5295388fafa7ed1d0011f6e9f536214" Jul 29 14:43:39.097688 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.097672 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh" Jul 29 14:43:39.097969 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.097942 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lllw7" podUID="7e74dacb-497f-4f6a-9c08-22c0fa028cd1" containerName="manager" containerID="cri-o://54d139fcb4d0e2da3d50137d39fbd003ae8f203cb33a395494f6871705da6f6b" gracePeriod=2 Jul 29 14:43:39.098898 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.098878 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" Jul 29 14:43:39.099528 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:43:39.099507 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-ntvrh\" is forbidden: User \"system:node:ip-10-0-132-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-42.ec2.internal' and this object" podUID="15cd006e-41be-4387-b3cc-846de47f2e24" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh" Jul 29 14:43:39.100778 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:43:39.100753 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lrv45\" is forbidden: User \"system:node:ip-10-0-132-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-42.ec2.internal' and this object" podUID="1aab703d-a990-4166-a5ae-97b7f3eac0e1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" Jul 29 14:43:39.102100 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:43:39.102074 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-ntvrh\" is forbidden: User \"system:node:ip-10-0-132-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-42.ec2.internal' and this object" podUID="15cd006e-41be-4387-b3cc-846de47f2e24" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh" Jul 29 14:43:39.103383 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:43:39.103356 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lrv45\" is forbidden: User \"system:node:ip-10-0-132-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-42.ec2.internal' and this object" podUID="1aab703d-a990-4166-a5ae-97b7f3eac0e1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" Jul 29 14:43:39.103904 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.103890 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh" Jul 29 14:43:39.104512 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:43:39.104495 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-lllw7\" is forbidden: User \"system:node:ip-10-0-132-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-42.ec2.internal' and this object" podUID="7e74dacb-497f-4f6a-9c08-22c0fa028cd1" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lllw7" Jul 29 14:43:39.105598 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:43:39.105577 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-ntvrh\" is forbidden: User \"system:node:ip-10-0-132-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-42.ec2.internal' and this object" podUID="15cd006e-41be-4387-b3cc-846de47f2e24" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh" Jul 29 14:43:39.106676 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:43:39.106654 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lrv45\" is forbidden: User \"system:node:ip-10-0-132-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-42.ec2.internal' and this object" podUID="1aab703d-a990-4166-a5ae-97b7f3eac0e1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" Jul 29 14:43:39.107820 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:43:39.107801 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-lllw7\" is forbidden: User \"system:node:ip-10-0-132-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-42.ec2.internal' and this object" podUID="7e74dacb-497f-4f6a-9c08-22c0fa028cd1" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lllw7" Jul 29 14:43:39.126108 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.126091 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwvx4\" (UniqueName: \"kubernetes.io/projected/15cd006e-41be-4387-b3cc-846de47f2e24-kube-api-access-bwvx4\") pod \"15cd006e-41be-4387-b3cc-846de47f2e24\" (UID: \"15cd006e-41be-4387-b3cc-846de47f2e24\") " Jul 29 14:43:39.126188 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.126126 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjpsh\" (UniqueName: \"kubernetes.io/projected/1aab703d-a990-4166-a5ae-97b7f3eac0e1-kube-api-access-fjpsh\") pod \"1aab703d-a990-4166-a5ae-97b7f3eac0e1\" (UID: \"1aab703d-a990-4166-a5ae-97b7f3eac0e1\") " Jul 29 14:43:39.126188 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.126163 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1aab703d-a990-4166-a5ae-97b7f3eac0e1-extensions-socket-volume\") pod \"1aab703d-a990-4166-a5ae-97b7f3eac0e1\" (UID: \"1aab703d-a990-4166-a5ae-97b7f3eac0e1\") " Jul 29 14:43:39.126279 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.126261 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15cd006e-41be-4387-b3cc-846de47f2e24-extensions-socket-volume\") pod \"15cd006e-41be-4387-b3cc-846de47f2e24\" (UID: \"15cd006e-41be-4387-b3cc-846de47f2e24\") " Jul 29 14:43:39.126489 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.126470 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15cd006e-41be-4387-b3cc-846de47f2e24-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "15cd006e-41be-4387-b3cc-846de47f2e24" (UID: "15cd006e-41be-4387-b3cc-846de47f2e24"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:43:39.126588 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.126575 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/15cd006e-41be-4387-b3cc-846de47f2e24-extensions-socket-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:39.126645 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.126572 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aab703d-a990-4166-a5ae-97b7f3eac0e1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1aab703d-a990-4166-a5ae-97b7f3eac0e1" (UID: "1aab703d-a990-4166-a5ae-97b7f3eac0e1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:43:39.128009 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.127988 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aab703d-a990-4166-a5ae-97b7f3eac0e1-kube-api-access-fjpsh" (OuterVolumeSpecName: "kube-api-access-fjpsh") pod "1aab703d-a990-4166-a5ae-97b7f3eac0e1" (UID: "1aab703d-a990-4166-a5ae-97b7f3eac0e1"). InnerVolumeSpecName "kube-api-access-fjpsh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 14:43:39.128105 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.128084 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15cd006e-41be-4387-b3cc-846de47f2e24-kube-api-access-bwvx4" (OuterVolumeSpecName: "kube-api-access-bwvx4") pod "15cd006e-41be-4387-b3cc-846de47f2e24" (UID: "15cd006e-41be-4387-b3cc-846de47f2e24"). InnerVolumeSpecName "kube-api-access-bwvx4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 14:43:39.160401 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.160353 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc" Jul 29 14:43:39.227295 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.227267 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwvx4\" (UniqueName: \"kubernetes.io/projected/15cd006e-41be-4387-b3cc-846de47f2e24-kube-api-access-bwvx4\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:39.227399 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.227301 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjpsh\" (UniqueName: \"kubernetes.io/projected/1aab703d-a990-4166-a5ae-97b7f3eac0e1-kube-api-access-fjpsh\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:39.227399 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.227317 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1aab703d-a990-4166-a5ae-97b7f3eac0e1-extensions-socket-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:39.236829 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.236803 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk" Jul 29 14:43:39.297853 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.297689 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc"] Jul 29 14:43:39.299365 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:43:39.299334 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebd67da2_8a88_43d6_9e62_b1d67553fae1.slice/crio-71c4317d9d5a381d34521b199d5e1be174ed815e3ef1fc2a7418cf7a9658ed0c WatchSource:0}: Error finding container 71c4317d9d5a381d34521b199d5e1be174ed815e3ef1fc2a7418cf7a9658ed0c: Status 404 returned error can't find the container with id 71c4317d9d5a381d34521b199d5e1be174ed815e3ef1fc2a7418cf7a9658ed0c Jul 29 14:43:39.359512 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.359490 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lllw7" Jul 29 14:43:39.362291 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:43:39.362266 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-ntvrh\" is forbidden: User \"system:node:ip-10-0-132-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-42.ec2.internal' and this object" podUID="15cd006e-41be-4387-b3cc-846de47f2e24" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh" Jul 29 14:43:39.364190 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:43:39.364169 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lrv45\" is forbidden: User \"system:node:ip-10-0-132-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-42.ec2.internal' and this object" podUID="1aab703d-a990-4166-a5ae-97b7f3eac0e1" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" Jul 29 14:43:39.365542 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:43:39.365519 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-lllw7\" is forbidden: User \"system:node:ip-10-0-132-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-42.ec2.internal' and this object" podUID="7e74dacb-497f-4f6a-9c08-22c0fa028cd1" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lllw7" Jul 29 14:43:39.385131 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.385022 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk"] Jul 29 14:43:39.387802 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:43:39.387776 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08d58c35_37e9_4d28_9b7f_aa44abd525b3.slice/crio-e8da6bf9c4be9deadebe06b99e9c31f544ec557c8276eaa70bda911affe6742e WatchSource:0}: Error finding container e8da6bf9c4be9deadebe06b99e9c31f544ec557c8276eaa70bda911affe6742e: Status 404 returned error can't find the container with id e8da6bf9c4be9deadebe06b99e9c31f544ec557c8276eaa70bda911affe6742e Jul 29 14:43:39.429683 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.429666 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ggqz\" (UniqueName: \"kubernetes.io/projected/7e74dacb-497f-4f6a-9c08-22c0fa028cd1-kube-api-access-7ggqz\") pod \"7e74dacb-497f-4f6a-9c08-22c0fa028cd1\" (UID: \"7e74dacb-497f-4f6a-9c08-22c0fa028cd1\") " Jul 29 14:43:39.429782 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.429698 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e74dacb-497f-4f6a-9c08-22c0fa028cd1-extensions-socket-volume\") pod \"7e74dacb-497f-4f6a-9c08-22c0fa028cd1\" (UID: \"7e74dacb-497f-4f6a-9c08-22c0fa028cd1\") " Jul 29 14:43:39.429994 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.429968 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e74dacb-497f-4f6a-9c08-22c0fa028cd1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7e74dacb-497f-4f6a-9c08-22c0fa028cd1" (UID: "7e74dacb-497f-4f6a-9c08-22c0fa028cd1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:43:39.431712 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.431690 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e74dacb-497f-4f6a-9c08-22c0fa028cd1-kube-api-access-7ggqz" (OuterVolumeSpecName: "kube-api-access-7ggqz") pod "7e74dacb-497f-4f6a-9c08-22c0fa028cd1" (UID: "7e74dacb-497f-4f6a-9c08-22c0fa028cd1"). InnerVolumeSpecName "kube-api-access-7ggqz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 14:43:39.530965 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.530942 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ggqz\" (UniqueName: \"kubernetes.io/projected/7e74dacb-497f-4f6a-9c08-22c0fa028cd1-kube-api-access-7ggqz\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:39.530965 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.530964 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e74dacb-497f-4f6a-9c08-22c0fa028cd1-extensions-socket-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:39.818271 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.818240 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15cd006e-41be-4387-b3cc-846de47f2e24" path="/var/lib/kubelet/pods/15cd006e-41be-4387-b3cc-846de47f2e24/volumes" Jul 29 14:43:39.818531 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.818516 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1aab703d-a990-4166-a5ae-97b7f3eac0e1" path="/var/lib/kubelet/pods/1aab703d-a990-4166-a5ae-97b7f3eac0e1/volumes" Jul 29 14:43:39.818881 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:39.818868 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e74dacb-497f-4f6a-9c08-22c0fa028cd1" path="/var/lib/kubelet/pods/7e74dacb-497f-4f6a-9c08-22c0fa028cd1/volumes" Jul 29 14:43:40.104677 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:40.104597 2580 generic.go:356] "Generic (PLEG): container finished" podID="7e74dacb-497f-4f6a-9c08-22c0fa028cd1" containerID="54d139fcb4d0e2da3d50137d39fbd003ae8f203cb33a395494f6871705da6f6b" exitCode=2 Jul 29 14:43:40.104677 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:40.104645 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lllw7" Jul 29 14:43:40.105152 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:40.104702 2580 scope.go:117] "RemoveContainer" containerID="54d139fcb4d0e2da3d50137d39fbd003ae8f203cb33a395494f6871705da6f6b" Jul 29 14:43:40.106765 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:40.106744 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc" event={"ID":"ebd67da2-8a88-43d6-9e62-b1d67553fae1","Type":"ContainerStarted","Data":"b06aba0e3fe6991acb73a128d2df25063776547d92ae709096abe52da7a97899"} Jul 29 14:43:40.106893 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:40.106794 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc" event={"ID":"ebd67da2-8a88-43d6-9e62-b1d67553fae1","Type":"ContainerStarted","Data":"71c4317d9d5a381d34521b199d5e1be174ed815e3ef1fc2a7418cf7a9658ed0c"} Jul 29 14:43:40.106893 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:40.106839 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc" Jul 29 14:43:40.108610 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:40.108586 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk" event={"ID":"08d58c35-37e9-4d28-9b7f-aa44abd525b3","Type":"ContainerStarted","Data":"febcb949abe7b58060de0486f62a36db89e82f16cc8beb25c4e8773d13356194"} Jul 29 14:43:40.108699 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:40.108611 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh" Jul 29 14:43:40.108699 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:40.108614 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrv45" Jul 29 14:43:40.108699 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:40.108643 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk" event={"ID":"08d58c35-37e9-4d28-9b7f-aa44abd525b3","Type":"ContainerStarted","Data":"e8da6bf9c4be9deadebe06b99e9c31f544ec557c8276eaa70bda911affe6742e"} Jul 29 14:43:40.108867 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:40.108830 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk" Jul 29 14:43:40.115073 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:40.115057 2580 scope.go:117] "RemoveContainer" containerID="54d139fcb4d0e2da3d50137d39fbd003ae8f203cb33a395494f6871705da6f6b" Jul 29 14:43:40.115322 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:43:40.115301 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54d139fcb4d0e2da3d50137d39fbd003ae8f203cb33a395494f6871705da6f6b\": container with ID starting with 54d139fcb4d0e2da3d50137d39fbd003ae8f203cb33a395494f6871705da6f6b not found: ID does not exist" containerID="54d139fcb4d0e2da3d50137d39fbd003ae8f203cb33a395494f6871705da6f6b" Jul 29 14:43:40.115399 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:40.115327 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54d139fcb4d0e2da3d50137d39fbd003ae8f203cb33a395494f6871705da6f6b"} err="failed to get container status \"54d139fcb4d0e2da3d50137d39fbd003ae8f203cb33a395494f6871705da6f6b\": rpc error: code = NotFound desc = could not find container \"54d139fcb4d0e2da3d50137d39fbd003ae8f203cb33a395494f6871705da6f6b\": container with ID starting with 54d139fcb4d0e2da3d50137d39fbd003ae8f203cb33a395494f6871705da6f6b not found: ID does not exist" Jul 29 14:43:40.131753 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:43:40.128881 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-ntvrh\" is forbidden: User \"system:node:ip-10-0-132-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-42.ec2.internal' and this object" podUID="15cd006e-41be-4387-b3cc-846de47f2e24" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh" Jul 29 14:43:40.131753 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:40.130193 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc" podStartSLOduration=2.130177751 podStartE2EDuration="2.130177751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:43:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:43:40.1271204 +0000 UTC m=+496.855269590" watchObservedRunningTime="2026-07-29 14:43:40.130177751 +0000 UTC m=+496.858326943" Jul 29 14:43:40.145147 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:40.145105 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk" podStartSLOduration=2.14509229 podStartE2EDuration="2.14509229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:43:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:43:40.144663771 +0000 UTC m=+496.872812977" watchObservedRunningTime="2026-07-29 14:43:40.14509229 +0000 UTC m=+496.873241479" Jul 29 14:43:40.146495 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:43:40.146471 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-lllw7\" is forbidden: User \"system:node:ip-10-0-132-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-42.ec2.internal' and this object" podUID="7e74dacb-497f-4f6a-9c08-22c0fa028cd1" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lllw7" Jul 29 14:43:41.057739 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.057676 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-69467bccf7-cpbq6" podUID="d3b0ea3f-aa66-468d-8fa1-42b1268935c2" containerName="console" containerID="cri-o://1fc1547fbbb5188982ffa30fb30ea6c295be88266c691507290508c12f1bf2cb" gracePeriod=15 Jul 29 14:43:41.309800 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.309734 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:43:41.311671 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:43:41.311633 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-ntvrh\" is forbidden: User \"system:node:ip-10-0-132-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-42.ec2.internal' and this object" podUID="15cd006e-41be-4387-b3cc-846de47f2e24" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-ntvrh" Jul 29 14:43:41.312990 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:43:41.312963 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-lllw7\" is forbidden: User \"system:node:ip-10-0-132-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-42.ec2.internal' and this object" podUID="7e74dacb-497f-4f6a-9c08-22c0fa028cd1" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-lllw7" Jul 29 14:43:41.344592 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.344565 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-console-serving-cert\") pod \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " Jul 29 14:43:41.344693 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.344616 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rwzp\" (UniqueName: \"kubernetes.io/projected/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-kube-api-access-5rwzp\") pod \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " Jul 29 14:43:41.344693 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.344664 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-console-oauth-config\") pod \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " Jul 29 14:43:41.344831 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.344692 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-console-config\") pod \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " Jul 29 14:43:41.344831 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.344799 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-service-ca\") pod \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " Jul 29 14:43:41.344933 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.344839 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-oauth-serving-cert\") pod \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " Jul 29 14:43:41.344933 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.344873 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-trusted-ca-bundle\") pod \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\" (UID: \"d3b0ea3f-aa66-468d-8fa1-42b1268935c2\") " Jul 29 14:43:41.345494 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.345092 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-console-config" (OuterVolumeSpecName: "console-config") pod "d3b0ea3f-aa66-468d-8fa1-42b1268935c2" (UID: "d3b0ea3f-aa66-468d-8fa1-42b1268935c2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 14:43:41.345494 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.345185 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-console-config\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:41.345494 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.345353 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-service-ca" (OuterVolumeSpecName: "service-ca") pod "d3b0ea3f-aa66-468d-8fa1-42b1268935c2" (UID: "d3b0ea3f-aa66-468d-8fa1-42b1268935c2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 14:43:41.345669 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.345489 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d3b0ea3f-aa66-468d-8fa1-42b1268935c2" (UID: "d3b0ea3f-aa66-468d-8fa1-42b1268935c2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 14:43:41.345669 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.345546 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d3b0ea3f-aa66-468d-8fa1-42b1268935c2" (UID: "d3b0ea3f-aa66-468d-8fa1-42b1268935c2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 14:43:41.347308 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.347262 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d3b0ea3f-aa66-468d-8fa1-42b1268935c2" (UID: "d3b0ea3f-aa66-468d-8fa1-42b1268935c2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 14:43:41.347410 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.347341 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-kube-api-access-5rwzp" (OuterVolumeSpecName: "kube-api-access-5rwzp") pod "d3b0ea3f-aa66-468d-8fa1-42b1268935c2" (UID: "d3b0ea3f-aa66-468d-8fa1-42b1268935c2"). InnerVolumeSpecName "kube-api-access-5rwzp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 14:43:41.347410 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.347349 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d3b0ea3f-aa66-468d-8fa1-42b1268935c2" (UID: "d3b0ea3f-aa66-468d-8fa1-42b1268935c2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 14:43:41.446394 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.446374 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-console-oauth-config\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:41.446394 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.446394 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-service-ca\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:41.446521 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.446403 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-oauth-serving-cert\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:41.446521 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.446411 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-trusted-ca-bundle\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:41.446521 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.446419 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-console-serving-cert\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:41.446521 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:41.446428 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rwzp\" (UniqueName: \"kubernetes.io/projected/d3b0ea3f-aa66-468d-8fa1-42b1268935c2-kube-api-access-5rwzp\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:42.121506 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:42.121437 2580 generic.go:356] "Generic (PLEG): container finished" podID="d3b0ea3f-aa66-468d-8fa1-42b1268935c2" containerID="1fc1547fbbb5188982ffa30fb30ea6c295be88266c691507290508c12f1bf2cb" exitCode=0 Jul 29 14:43:42.121506 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:42.121487 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69467bccf7-cpbq6" Jul 29 14:43:42.121644 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:42.121530 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69467bccf7-cpbq6" event={"ID":"d3b0ea3f-aa66-468d-8fa1-42b1268935c2","Type":"ContainerDied","Data":"1fc1547fbbb5188982ffa30fb30ea6c295be88266c691507290508c12f1bf2cb"} Jul 29 14:43:42.121644 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:42.121583 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69467bccf7-cpbq6" event={"ID":"d3b0ea3f-aa66-468d-8fa1-42b1268935c2","Type":"ContainerDied","Data":"8adb8668217c472fb070e76b554edf67076775c5fbd8fb2a7a72a286d70bb39b"} Jul 29 14:43:42.121644 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:42.121597 2580 scope.go:117] "RemoveContainer" containerID="1fc1547fbbb5188982ffa30fb30ea6c295be88266c691507290508c12f1bf2cb" Jul 29 14:43:42.132260 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:42.132239 2580 scope.go:117] "RemoveContainer" containerID="1fc1547fbbb5188982ffa30fb30ea6c295be88266c691507290508c12f1bf2cb" Jul 29 14:43:42.132556 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:43:42.132538 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fc1547fbbb5188982ffa30fb30ea6c295be88266c691507290508c12f1bf2cb\": container with ID starting with 1fc1547fbbb5188982ffa30fb30ea6c295be88266c691507290508c12f1bf2cb not found: ID does not exist" containerID="1fc1547fbbb5188982ffa30fb30ea6c295be88266c691507290508c12f1bf2cb" Jul 29 14:43:42.132635 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:42.132563 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fc1547fbbb5188982ffa30fb30ea6c295be88266c691507290508c12f1bf2cb"} err="failed to get container status \"1fc1547fbbb5188982ffa30fb30ea6c295be88266c691507290508c12f1bf2cb\": rpc error: code = NotFound desc = could not find container \"1fc1547fbbb5188982ffa30fb30ea6c295be88266c691507290508c12f1bf2cb\": container with ID starting with 1fc1547fbbb5188982ffa30fb30ea6c295be88266c691507290508c12f1bf2cb not found: ID does not exist" Jul 29 14:43:42.148540 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:42.148515 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69467bccf7-cpbq6"] Jul 29 14:43:42.152212 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:42.152193 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-69467bccf7-cpbq6"] Jul 29 14:43:43.423826 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:43.423784 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:43.523963 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:43.523937 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:43.819708 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:43.819682 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3b0ea3f-aa66-468d-8fa1-42b1268935c2" path="/var/lib/kubelet/pods/d3b0ea3f-aa66-468d-8fa1-42b1268935c2/volumes" Jul 29 14:43:44.025511 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:44.025458 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:44.101533 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:44.101463 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:47.420219 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:47.420159 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:47.521131 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:47.521099 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:48.595866 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:48.595826 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:51.116967 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.116926 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc" Jul 29 14:43:51.117375 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.117132 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk" Jul 29 14:43:51.200068 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.200033 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk"] Jul 29 14:43:51.200228 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.200197 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk" podUID="08d58c35-37e9-4d28-9b7f-aa44abd525b3" containerName="manager" containerID="cri-o://febcb949abe7b58060de0486f62a36db89e82f16cc8beb25c4e8773d13356194" gracePeriod=10 Jul 29 14:43:51.257266 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.257239 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:51.358018 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.357978 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:51.430358 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.430322 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg"] Jul 29 14:43:51.430760 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.430744 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3b0ea3f-aa66-468d-8fa1-42b1268935c2" containerName="console" Jul 29 14:43:51.430760 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.430760 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3b0ea3f-aa66-468d-8fa1-42b1268935c2" containerName="console" Jul 29 14:43:51.430891 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.430835 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3b0ea3f-aa66-468d-8fa1-42b1268935c2" containerName="console" Jul 29 14:43:51.434661 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.434640 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg" Jul 29 14:43:51.451886 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.451864 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg"] Jul 29 14:43:51.452632 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.452613 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk" Jul 29 14:43:51.520544 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.520521 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08d58c35-37e9-4d28-9b7f-aa44abd525b3-extensions-socket-volume\") pod \"08d58c35-37e9-4d28-9b7f-aa44abd525b3\" (UID: \"08d58c35-37e9-4d28-9b7f-aa44abd525b3\") " Jul 29 14:43:51.520636 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.520601 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-js8b7\" (UniqueName: \"kubernetes.io/projected/08d58c35-37e9-4d28-9b7f-aa44abd525b3-kube-api-access-js8b7\") pod \"08d58c35-37e9-4d28-9b7f-aa44abd525b3\" (UID: \"08d58c35-37e9-4d28-9b7f-aa44abd525b3\") " Jul 29 14:43:51.520777 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.520758 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ggl9\" (UniqueName: \"kubernetes.io/projected/1914971c-3548-4f9f-8c83-80b25eb4f188-kube-api-access-6ggl9\") pod \"kuadrant-operator-controller-manager-86878c769f-62cwg\" (UID: \"1914971c-3548-4f9f-8c83-80b25eb4f188\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg" Jul 29 14:43:51.520881 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.520860 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1914971c-3548-4f9f-8c83-80b25eb4f188-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-62cwg\" (UID: \"1914971c-3548-4f9f-8c83-80b25eb4f188\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg" Jul 29 14:43:51.520935 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.520875 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08d58c35-37e9-4d28-9b7f-aa44abd525b3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "08d58c35-37e9-4d28-9b7f-aa44abd525b3" (UID: "08d58c35-37e9-4d28-9b7f-aa44abd525b3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:43:51.520988 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.520943 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08d58c35-37e9-4d28-9b7f-aa44abd525b3-extensions-socket-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:51.522580 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.522556 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08d58c35-37e9-4d28-9b7f-aa44abd525b3-kube-api-access-js8b7" (OuterVolumeSpecName: "kube-api-access-js8b7") pod "08d58c35-37e9-4d28-9b7f-aa44abd525b3" (UID: "08d58c35-37e9-4d28-9b7f-aa44abd525b3"). InnerVolumeSpecName "kube-api-access-js8b7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 14:43:51.621465 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.621412 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ggl9\" (UniqueName: \"kubernetes.io/projected/1914971c-3548-4f9f-8c83-80b25eb4f188-kube-api-access-6ggl9\") pod \"kuadrant-operator-controller-manager-86878c769f-62cwg\" (UID: \"1914971c-3548-4f9f-8c83-80b25eb4f188\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg" Jul 29 14:43:51.621465 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.621457 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1914971c-3548-4f9f-8c83-80b25eb4f188-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-62cwg\" (UID: \"1914971c-3548-4f9f-8c83-80b25eb4f188\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg" Jul 29 14:43:51.621595 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.621501 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-js8b7\" (UniqueName: \"kubernetes.io/projected/08d58c35-37e9-4d28-9b7f-aa44abd525b3-kube-api-access-js8b7\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:43:51.621768 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.621754 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1914971c-3548-4f9f-8c83-80b25eb4f188-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-62cwg\" (UID: \"1914971c-3548-4f9f-8c83-80b25eb4f188\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg" Jul 29 14:43:51.633502 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.633475 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ggl9\" (UniqueName: \"kubernetes.io/projected/1914971c-3548-4f9f-8c83-80b25eb4f188-kube-api-access-6ggl9\") pod \"kuadrant-operator-controller-manager-86878c769f-62cwg\" (UID: \"1914971c-3548-4f9f-8c83-80b25eb4f188\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg" Jul 29 14:43:51.758815 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.758790 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg" Jul 29 14:43:51.890813 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:51.890785 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg"] Jul 29 14:43:51.892526 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:43:51.892500 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1914971c_3548_4f9f_8c83_80b25eb4f188.slice/crio-12e41fae0c316f36a8aa4e7d0b10978126dfb2b17061725c2b1fe8131a85c4b3 WatchSource:0}: Error finding container 12e41fae0c316f36a8aa4e7d0b10978126dfb2b17061725c2b1fe8131a85c4b3: Status 404 returned error can't find the container with id 12e41fae0c316f36a8aa4e7d0b10978126dfb2b17061725c2b1fe8131a85c4b3 Jul 29 14:43:52.180902 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:52.180816 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg" event={"ID":"1914971c-3548-4f9f-8c83-80b25eb4f188","Type":"ContainerStarted","Data":"c0b24ab01cf58ecaa9c2b7fa6041aff24161e097a26ddaa5f6d4bf440447dc91"} Jul 29 14:43:52.180902 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:52.180885 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg" event={"ID":"1914971c-3548-4f9f-8c83-80b25eb4f188","Type":"ContainerStarted","Data":"12e41fae0c316f36a8aa4e7d0b10978126dfb2b17061725c2b1fe8131a85c4b3"} Jul 29 14:43:52.181327 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:52.180920 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg" Jul 29 14:43:52.182147 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:52.182119 2580 generic.go:356] "Generic (PLEG): container finished" podID="08d58c35-37e9-4d28-9b7f-aa44abd525b3" containerID="febcb949abe7b58060de0486f62a36db89e82f16cc8beb25c4e8773d13356194" exitCode=0 Jul 29 14:43:52.182232 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:52.182159 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk" Jul 29 14:43:52.182232 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:52.182165 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk" event={"ID":"08d58c35-37e9-4d28-9b7f-aa44abd525b3","Type":"ContainerDied","Data":"febcb949abe7b58060de0486f62a36db89e82f16cc8beb25c4e8773d13356194"} Jul 29 14:43:52.182232 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:52.182211 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk" event={"ID":"08d58c35-37e9-4d28-9b7f-aa44abd525b3","Type":"ContainerDied","Data":"e8da6bf9c4be9deadebe06b99e9c31f544ec557c8276eaa70bda911affe6742e"} Jul 29 14:43:52.182326 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:52.182241 2580 scope.go:117] "RemoveContainer" containerID="febcb949abe7b58060de0486f62a36db89e82f16cc8beb25c4e8773d13356194" Jul 29 14:43:52.190513 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:52.190495 2580 scope.go:117] "RemoveContainer" containerID="febcb949abe7b58060de0486f62a36db89e82f16cc8beb25c4e8773d13356194" Jul 29 14:43:52.190769 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:43:52.190747 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"febcb949abe7b58060de0486f62a36db89e82f16cc8beb25c4e8773d13356194\": container with ID starting with febcb949abe7b58060de0486f62a36db89e82f16cc8beb25c4e8773d13356194 not found: ID does not exist" containerID="febcb949abe7b58060de0486f62a36db89e82f16cc8beb25c4e8773d13356194" Jul 29 14:43:52.190857 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:52.190776 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"febcb949abe7b58060de0486f62a36db89e82f16cc8beb25c4e8773d13356194"} err="failed to get container status \"febcb949abe7b58060de0486f62a36db89e82f16cc8beb25c4e8773d13356194\": rpc error: code = NotFound desc = could not find container \"febcb949abe7b58060de0486f62a36db89e82f16cc8beb25c4e8773d13356194\": container with ID starting with febcb949abe7b58060de0486f62a36db89e82f16cc8beb25c4e8773d13356194 not found: ID does not exist" Jul 29 14:43:52.205975 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:52.205917 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg" podStartSLOduration=1.205904329 podStartE2EDuration="1.205904329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:43:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:43:52.204232779 +0000 UTC m=+508.932381967" watchObservedRunningTime="2026-07-29 14:43:52.205904329 +0000 UTC m=+508.934053588" Jul 29 14:43:52.219992 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:52.219972 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk"] Jul 29 14:43:52.227141 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:52.227119 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-x7nzk"] Jul 29 14:43:53.825595 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:53.825557 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08d58c35-37e9-4d28-9b7f-aa44abd525b3" path="/var/lib/kubelet/pods/08d58c35-37e9-4d28-9b7f-aa44abd525b3/volumes" Jul 29 14:43:56.544589 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:56.544556 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:56.865191 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:56.865098 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:56.957859 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:56.957830 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:59.751073 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:59.751011 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:43:59.851734 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:43:59.851680 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:44:03.192078 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:03.192044 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg" Jul 29 14:44:03.262757 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:03.262713 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc"] Jul 29 14:44:03.262934 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:03.262915 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc" podUID="ebd67da2-8a88-43d6-9e62-b1d67553fae1" containerName="manager" containerID="cri-o://b06aba0e3fe6991acb73a128d2df25063776547d92ae709096abe52da7a97899" gracePeriod=10 Jul 29 14:44:03.504970 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:03.504947 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc" Jul 29 14:44:03.608539 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:03.608510 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8drt\" (UniqueName: \"kubernetes.io/projected/ebd67da2-8a88-43d6-9e62-b1d67553fae1-kube-api-access-p8drt\") pod \"ebd67da2-8a88-43d6-9e62-b1d67553fae1\" (UID: \"ebd67da2-8a88-43d6-9e62-b1d67553fae1\") " Jul 29 14:44:03.608668 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:03.608586 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ebd67da2-8a88-43d6-9e62-b1d67553fae1-extensions-socket-volume\") pod \"ebd67da2-8a88-43d6-9e62-b1d67553fae1\" (UID: \"ebd67da2-8a88-43d6-9e62-b1d67553fae1\") " Jul 29 14:44:03.609002 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:03.608979 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebd67da2-8a88-43d6-9e62-b1d67553fae1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ebd67da2-8a88-43d6-9e62-b1d67553fae1" (UID: "ebd67da2-8a88-43d6-9e62-b1d67553fae1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:44:03.610374 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:03.610353 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebd67da2-8a88-43d6-9e62-b1d67553fae1-kube-api-access-p8drt" (OuterVolumeSpecName: "kube-api-access-p8drt") pod "ebd67da2-8a88-43d6-9e62-b1d67553fae1" (UID: "ebd67da2-8a88-43d6-9e62-b1d67553fae1"). InnerVolumeSpecName "kube-api-access-p8drt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 14:44:03.709108 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:03.709085 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ebd67da2-8a88-43d6-9e62-b1d67553fae1-extensions-socket-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:44:03.709108 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:03.709105 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8drt\" (UniqueName: \"kubernetes.io/projected/ebd67da2-8a88-43d6-9e62-b1d67553fae1-kube-api-access-p8drt\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:44:04.251520 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:04.251489 2580 generic.go:356] "Generic (PLEG): container finished" podID="ebd67da2-8a88-43d6-9e62-b1d67553fae1" containerID="b06aba0e3fe6991acb73a128d2df25063776547d92ae709096abe52da7a97899" exitCode=0 Jul 29 14:44:04.251885 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:04.251532 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc" Jul 29 14:44:04.251885 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:04.251532 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc" event={"ID":"ebd67da2-8a88-43d6-9e62-b1d67553fae1","Type":"ContainerDied","Data":"b06aba0e3fe6991acb73a128d2df25063776547d92ae709096abe52da7a97899"} Jul 29 14:44:04.251885 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:04.251590 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc" event={"ID":"ebd67da2-8a88-43d6-9e62-b1d67553fae1","Type":"ContainerDied","Data":"71c4317d9d5a381d34521b199d5e1be174ed815e3ef1fc2a7418cf7a9658ed0c"} Jul 29 14:44:04.251885 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:04.251615 2580 scope.go:117] "RemoveContainer" containerID="b06aba0e3fe6991acb73a128d2df25063776547d92ae709096abe52da7a97899" Jul 29 14:44:04.261028 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:04.261013 2580 scope.go:117] "RemoveContainer" containerID="b06aba0e3fe6991acb73a128d2df25063776547d92ae709096abe52da7a97899" Jul 29 14:44:04.261332 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:44:04.261309 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b06aba0e3fe6991acb73a128d2df25063776547d92ae709096abe52da7a97899\": container with ID starting with b06aba0e3fe6991acb73a128d2df25063776547d92ae709096abe52da7a97899 not found: ID does not exist" containerID="b06aba0e3fe6991acb73a128d2df25063776547d92ae709096abe52da7a97899" Jul 29 14:44:04.261431 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:04.261340 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b06aba0e3fe6991acb73a128d2df25063776547d92ae709096abe52da7a97899"} err="failed to get container status \"b06aba0e3fe6991acb73a128d2df25063776547d92ae709096abe52da7a97899\": rpc error: code = NotFound desc = could not find container \"b06aba0e3fe6991acb73a128d2df25063776547d92ae709096abe52da7a97899\": container with ID starting with b06aba0e3fe6991acb73a128d2df25063776547d92ae709096abe52da7a97899 not found: ID does not exist" Jul 29 14:44:04.272313 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:04.272286 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc"] Jul 29 14:44:04.281751 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:04.281714 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-w86wc"] Jul 29 14:44:05.817440 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:05.817403 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebd67da2-8a88-43d6-9e62-b1d67553fae1" path="/var/lib/kubelet/pods/ebd67da2-8a88-43d6-9e62-b1d67553fae1/volumes" Jul 29 14:44:07.746175 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:07.746142 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:44:08.147610 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:08.147490 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:44:08.326693 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:08.326655 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:44:15.441918 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:15.441799 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:44:15.543039 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:15.543007 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:44:22.211154 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:22.211118 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:44:22.326382 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:22.326339 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:44:22.380742 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:22.380683 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:44:23.071668 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:23.071622 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:44:23.141124 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:23.141092 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:44:41.059345 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:41.059293 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:44:41.161432 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:41.161400 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:44:41.594753 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:41.594700 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:44:53.464661 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:53.464628 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:44:53.563346 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:53.563287 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:44:59.948006 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:44:59.947964 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:45:00.051352 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:00.051321 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:45:00.150593 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:00.150556 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:45:03.872513 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:03.872478 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:45:03.990316 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:03.990275 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:45:23.699325 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:23.699292 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 14:45:23.704202 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:23.704180 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 14:45:35.625213 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:35.625169 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:45:36.980573 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:36.980546 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-g5ttp"] Jul 29 14:45:36.980939 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:36.980909 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebd67da2-8a88-43d6-9e62-b1d67553fae1" containerName="manager" Jul 29 14:45:36.980939 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:36.980921 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebd67da2-8a88-43d6-9e62-b1d67553fae1" containerName="manager" Jul 29 14:45:36.981015 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:36.980942 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08d58c35-37e9-4d28-9b7f-aa44abd525b3" containerName="manager" Jul 29 14:45:36.981015 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:36.980948 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="08d58c35-37e9-4d28-9b7f-aa44abd525b3" containerName="manager" Jul 29 14:45:36.981015 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:36.980993 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebd67da2-8a88-43d6-9e62-b1d67553fae1" containerName="manager" Jul 29 14:45:36.981015 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:36.981004 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="08d58c35-37e9-4d28-9b7f-aa44abd525b3" containerName="manager" Jul 29 14:45:36.983229 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:36.983209 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-g5ttp" Jul 29 14:45:36.986625 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:36.986602 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 29 14:45:36.994633 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:36.994610 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-g5ttp"] Jul 29 14:45:37.015835 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.015809 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-45bsl"] Jul 29 14:45:37.021258 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.021225 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-45bsl" Jul 29 14:45:37.023605 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.023580 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-fhx9f\"" Jul 29 14:45:37.023984 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.023955 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq767\" (UniqueName: \"kubernetes.io/projected/8fb9e0c2-fa8a-418c-94c8-94f7c8222b21-kube-api-access-fq767\") pod \"payload-pre-processing-687dbb6cf6-g5ttp\" (UID: \"8fb9e0c2-fa8a-418c-94c8-94f7c8222b21\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-g5ttp" Jul 29 14:45:37.024637 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.024564 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fb9e0c2-fa8a-418c-94c8-94f7c8222b21-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-g5ttp\" (UID: \"8fb9e0c2-fa8a-418c-94c8-94f7c8222b21\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-g5ttp" Jul 29 14:45:37.035459 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.035437 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-45bsl"] Jul 29 14:45:37.126009 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.125973 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fb9e0c2-fa8a-418c-94c8-94f7c8222b21-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-g5ttp\" (UID: \"8fb9e0c2-fa8a-418c-94c8-94f7c8222b21\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-g5ttp" Jul 29 14:45:37.126178 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.126032 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1695ee5c-3411-48e2-85e7-f11d527f6f5e-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-45bsl\" (UID: \"1695ee5c-3411-48e2-85e7-f11d527f6f5e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-45bsl" Jul 29 14:45:37.126178 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.126127 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29cqm\" (UniqueName: \"kubernetes.io/projected/1695ee5c-3411-48e2-85e7-f11d527f6f5e-kube-api-access-29cqm\") pod \"payload-processing-5cc6cd444b-45bsl\" (UID: \"1695ee5c-3411-48e2-85e7-f11d527f6f5e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-45bsl" Jul 29 14:45:37.126178 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.126161 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq767\" (UniqueName: \"kubernetes.io/projected/8fb9e0c2-fa8a-418c-94c8-94f7c8222b21-kube-api-access-fq767\") pod \"payload-pre-processing-687dbb6cf6-g5ttp\" (UID: \"8fb9e0c2-fa8a-418c-94c8-94f7c8222b21\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-g5ttp" Jul 29 14:45:37.126690 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.126674 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fb9e0c2-fa8a-418c-94c8-94f7c8222b21-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-g5ttp\" (UID: \"8fb9e0c2-fa8a-418c-94c8-94f7c8222b21\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-g5ttp" Jul 29 14:45:37.133378 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.133361 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq767\" (UniqueName: \"kubernetes.io/projected/8fb9e0c2-fa8a-418c-94c8-94f7c8222b21-kube-api-access-fq767\") pod \"payload-pre-processing-687dbb6cf6-g5ttp\" (UID: \"8fb9e0c2-fa8a-418c-94c8-94f7c8222b21\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-g5ttp" Jul 29 14:45:37.227062 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.227027 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29cqm\" (UniqueName: \"kubernetes.io/projected/1695ee5c-3411-48e2-85e7-f11d527f6f5e-kube-api-access-29cqm\") pod \"payload-processing-5cc6cd444b-45bsl\" (UID: \"1695ee5c-3411-48e2-85e7-f11d527f6f5e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-45bsl" Jul 29 14:45:37.227215 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.227094 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1695ee5c-3411-48e2-85e7-f11d527f6f5e-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-45bsl\" (UID: \"1695ee5c-3411-48e2-85e7-f11d527f6f5e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-45bsl" Jul 29 14:45:37.227701 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.227680 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1695ee5c-3411-48e2-85e7-f11d527f6f5e-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-45bsl\" (UID: \"1695ee5c-3411-48e2-85e7-f11d527f6f5e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-45bsl" Jul 29 14:45:37.234229 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.234170 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29cqm\" (UniqueName: \"kubernetes.io/projected/1695ee5c-3411-48e2-85e7-f11d527f6f5e-kube-api-access-29cqm\") pod \"payload-processing-5cc6cd444b-45bsl\" (UID: \"1695ee5c-3411-48e2-85e7-f11d527f6f5e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-45bsl" Jul 29 14:45:37.294490 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.294468 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-g5ttp" Jul 29 14:45:37.335028 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.334992 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-45bsl" Jul 29 14:45:37.446449 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.446421 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-g5ttp"] Jul 29 14:45:37.449122 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:45:37.449095 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fb9e0c2_fa8a_418c_94c8_94f7c8222b21.slice/crio-7549bec9a2024c660f8760739078070beda55ab865c297febdace3027318056b WatchSource:0}: Error finding container 7549bec9a2024c660f8760739078070beda55ab865c297febdace3027318056b: Status 404 returned error can't find the container with id 7549bec9a2024c660f8760739078070beda55ab865c297febdace3027318056b Jul 29 14:45:37.450982 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.450959 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 14:45:37.455087 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.454770 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:45:37.489969 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.489946 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-45bsl"] Jul 29 14:45:37.491302 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:45:37.491279 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1695ee5c_3411_48e2_85e7_f11d527f6f5e.slice/crio-b9d0f3e631e8a511c63c96f39496544e6520aebfcd7412a894e99a1057f2767b WatchSource:0}: Error finding container b9d0f3e631e8a511c63c96f39496544e6520aebfcd7412a894e99a1057f2767b: Status 404 returned error can't find the container with id b9d0f3e631e8a511c63c96f39496544e6520aebfcd7412a894e99a1057f2767b Jul 29 14:45:37.744518 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.744487 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-g5ttp" event={"ID":"8fb9e0c2-fa8a-418c-94c8-94f7c8222b21","Type":"ContainerStarted","Data":"7549bec9a2024c660f8760739078070beda55ab865c297febdace3027318056b"} Jul 29 14:45:37.745559 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:37.745540 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-45bsl" event={"ID":"1695ee5c-3411-48e2-85e7-f11d527f6f5e","Type":"ContainerStarted","Data":"b9d0f3e631e8a511c63c96f39496544e6520aebfcd7412a894e99a1057f2767b"} Jul 29 14:45:39.761935 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:39.761898 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-g5ttp" event={"ID":"8fb9e0c2-fa8a-418c-94c8-94f7c8222b21","Type":"ContainerStarted","Data":"384b0a90ecd0a74536e13eea149428c7c07e3a1d9f9676b3d5211a1c16d9422b"} Jul 29 14:45:39.762458 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:39.762000 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-g5ttp" Jul 29 14:45:39.763491 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:39.763465 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-45bsl" event={"ID":"1695ee5c-3411-48e2-85e7-f11d527f6f5e","Type":"ContainerStarted","Data":"b1bdb3e6a6972fc2440aedb39c9f2112e247a4f35e257413a5747056686dc2fc"} Jul 29 14:45:39.763583 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:39.763550 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-45bsl" Jul 29 14:45:39.784872 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:39.784832 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-g5ttp" podStartSLOduration=1.936687493 podStartE2EDuration="3.784821458s" totalImagesPullingTime="1.848133965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:45:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:45:39.78380201 +0000 UTC m=+616.511951219" watchObservedRunningTime="2026-07-29 14:45:39.784821458 +0000 UTC m=+616.512970647" Jul 29 14:45:39.803328 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:39.803290 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-45bsl" podStartSLOduration=2.001332518 podStartE2EDuration="3.80327702s" totalImagesPullingTime="1.801944502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:45:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:45:39.801972578 +0000 UTC m=+616.530121771" watchObservedRunningTime="2026-07-29 14:45:39.80327702 +0000 UTC m=+616.531426209" Jul 29 14:45:47.199848 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:47.199707 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:45:47.292181 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:47.292150 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:45:50.771705 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:50.771679 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-g5ttp" Jul 29 14:45:50.772174 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:45:50.771760 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-45bsl" Jul 29 14:46:05.013905 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:05.013866 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:46:06.637758 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:06.637701 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:46:11.304844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.304812 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd"] Jul 29 14:46:11.309089 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.309069 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:11.312322 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.312299 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 29 14:46:11.312527 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.312312 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6c2gw\"" Jul 29 14:46:11.312611 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.312363 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 29 14:46:11.312689 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.312441 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 29 14:46:11.316791 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.316770 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd"] Jul 29 14:46:11.360663 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.360638 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:46:11.402899 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.402867 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7eeb41e-6c1c-4621-b85f-90068071b442-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qzhd\" (UID: \"a7eeb41e-6c1c-4621-b85f-90068071b442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:11.403002 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.402908 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dx6f\" (UniqueName: \"kubernetes.io/projected/a7eeb41e-6c1c-4621-b85f-90068071b442-kube-api-access-5dx6f\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qzhd\" (UID: \"a7eeb41e-6c1c-4621-b85f-90068071b442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:11.403002 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.402961 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7eeb41e-6c1c-4621-b85f-90068071b442-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qzhd\" (UID: \"a7eeb41e-6c1c-4621-b85f-90068071b442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:11.403002 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.402992 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7eeb41e-6c1c-4621-b85f-90068071b442-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qzhd\" (UID: \"a7eeb41e-6c1c-4621-b85f-90068071b442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:11.403108 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.403039 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7eeb41e-6c1c-4621-b85f-90068071b442-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qzhd\" (UID: \"a7eeb41e-6c1c-4621-b85f-90068071b442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:11.403108 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.403054 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7eeb41e-6c1c-4621-b85f-90068071b442-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qzhd\" (UID: \"a7eeb41e-6c1c-4621-b85f-90068071b442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:11.503371 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.503346 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7eeb41e-6c1c-4621-b85f-90068071b442-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qzhd\" (UID: \"a7eeb41e-6c1c-4621-b85f-90068071b442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:11.503480 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.503375 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7eeb41e-6c1c-4621-b85f-90068071b442-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qzhd\" (UID: \"a7eeb41e-6c1c-4621-b85f-90068071b442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:11.503480 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.503408 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7eeb41e-6c1c-4621-b85f-90068071b442-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qzhd\" (UID: \"a7eeb41e-6c1c-4621-b85f-90068071b442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:11.503555 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.503512 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7eeb41e-6c1c-4621-b85f-90068071b442-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qzhd\" (UID: \"a7eeb41e-6c1c-4621-b85f-90068071b442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:11.503596 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.503579 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7eeb41e-6c1c-4621-b85f-90068071b442-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qzhd\" (UID: \"a7eeb41e-6c1c-4621-b85f-90068071b442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:11.503633 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.503618 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dx6f\" (UniqueName: \"kubernetes.io/projected/a7eeb41e-6c1c-4621-b85f-90068071b442-kube-api-access-5dx6f\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qzhd\" (UID: \"a7eeb41e-6c1c-4621-b85f-90068071b442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:11.503797 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.503775 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7eeb41e-6c1c-4621-b85f-90068071b442-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qzhd\" (UID: \"a7eeb41e-6c1c-4621-b85f-90068071b442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:11.503881 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.503859 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7eeb41e-6c1c-4621-b85f-90068071b442-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qzhd\" (UID: \"a7eeb41e-6c1c-4621-b85f-90068071b442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:11.503955 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.503936 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7eeb41e-6c1c-4621-b85f-90068071b442-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qzhd\" (UID: \"a7eeb41e-6c1c-4621-b85f-90068071b442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:11.505630 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.505611 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7eeb41e-6c1c-4621-b85f-90068071b442-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qzhd\" (UID: \"a7eeb41e-6c1c-4621-b85f-90068071b442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:11.505828 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.505811 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7eeb41e-6c1c-4621-b85f-90068071b442-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qzhd\" (UID: \"a7eeb41e-6c1c-4621-b85f-90068071b442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:11.511296 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.511276 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dx6f\" (UniqueName: \"kubernetes.io/projected/a7eeb41e-6c1c-4621-b85f-90068071b442-kube-api-access-5dx6f\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2qzhd\" (UID: \"a7eeb41e-6c1c-4621-b85f-90068071b442\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:11.622213 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.622151 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:11.752961 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.752933 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd"] Jul 29 14:46:11.755118 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:46:11.755090 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7eeb41e_6c1c_4621_b85f_90068071b442.slice/crio-207984e2a540d4f82e09acd6b844e4b84f77e3f8fda047a2b8988050f52fcf23 WatchSource:0}: Error finding container 207984e2a540d4f82e09acd6b844e4b84f77e3f8fda047a2b8988050f52fcf23: Status 404 returned error can't find the container with id 207984e2a540d4f82e09acd6b844e4b84f77e3f8fda047a2b8988050f52fcf23 Jul 29 14:46:11.931054 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:11.930960 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" event={"ID":"a7eeb41e-6c1c-4621-b85f-90068071b442","Type":"ContainerStarted","Data":"207984e2a540d4f82e09acd6b844e4b84f77e3f8fda047a2b8988050f52fcf23"} Jul 29 14:46:14.948064 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:14.948025 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" event={"ID":"a7eeb41e-6c1c-4621-b85f-90068071b442","Type":"ContainerStarted","Data":"60f174dad2350ca0d1b8e5272a3e19d01fd9c44043f001e09278c64a3b5014a4"} Jul 29 14:46:14.964540 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:14.964498 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" podStartSLOduration=0.890686691 podStartE2EDuration="3.9644841s" totalImagesPullingTime="3.073797409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:46:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:46:14.962712048 +0000 UTC m=+651.690861236" watchObservedRunningTime="2026-07-29 14:46:14.9644841 +0000 UTC m=+651.692633288" Jul 29 14:46:21.098208 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.098170 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr"] Jul 29 14:46:21.101930 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.101906 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:21.104634 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.104609 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 29 14:46:21.110097 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.110076 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr"] Jul 29 14:46:21.127612 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.127586 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:46:21.176262 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.176237 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/171f57a0-37f5-4b66-b8c7-dea91ddd1437-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wpnfr\" (UID: \"171f57a0-37f5-4b66-b8c7-dea91ddd1437\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:21.176364 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.176271 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/171f57a0-37f5-4b66-b8c7-dea91ddd1437-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wpnfr\" (UID: \"171f57a0-37f5-4b66-b8c7-dea91ddd1437\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:21.176364 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.176312 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tvcv\" (UniqueName: \"kubernetes.io/projected/171f57a0-37f5-4b66-b8c7-dea91ddd1437-kube-api-access-5tvcv\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wpnfr\" (UID: \"171f57a0-37f5-4b66-b8c7-dea91ddd1437\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:21.176445 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.176377 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/171f57a0-37f5-4b66-b8c7-dea91ddd1437-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wpnfr\" (UID: \"171f57a0-37f5-4b66-b8c7-dea91ddd1437\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:21.176445 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.176408 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/171f57a0-37f5-4b66-b8c7-dea91ddd1437-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wpnfr\" (UID: \"171f57a0-37f5-4b66-b8c7-dea91ddd1437\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:21.176445 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.176426 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/171f57a0-37f5-4b66-b8c7-dea91ddd1437-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wpnfr\" (UID: \"171f57a0-37f5-4b66-b8c7-dea91ddd1437\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:21.277217 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.277179 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/171f57a0-37f5-4b66-b8c7-dea91ddd1437-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wpnfr\" (UID: \"171f57a0-37f5-4b66-b8c7-dea91ddd1437\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:21.277217 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.277214 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/171f57a0-37f5-4b66-b8c7-dea91ddd1437-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wpnfr\" (UID: \"171f57a0-37f5-4b66-b8c7-dea91ddd1437\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:21.277409 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.277246 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tvcv\" (UniqueName: \"kubernetes.io/projected/171f57a0-37f5-4b66-b8c7-dea91ddd1437-kube-api-access-5tvcv\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wpnfr\" (UID: \"171f57a0-37f5-4b66-b8c7-dea91ddd1437\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:21.277409 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.277274 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/171f57a0-37f5-4b66-b8c7-dea91ddd1437-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wpnfr\" (UID: \"171f57a0-37f5-4b66-b8c7-dea91ddd1437\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:21.277409 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.277294 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/171f57a0-37f5-4b66-b8c7-dea91ddd1437-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wpnfr\" (UID: \"171f57a0-37f5-4b66-b8c7-dea91ddd1437\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:21.277409 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.277309 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/171f57a0-37f5-4b66-b8c7-dea91ddd1437-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wpnfr\" (UID: \"171f57a0-37f5-4b66-b8c7-dea91ddd1437\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:21.277635 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.277612 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/171f57a0-37f5-4b66-b8c7-dea91ddd1437-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wpnfr\" (UID: \"171f57a0-37f5-4b66-b8c7-dea91ddd1437\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:21.277696 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.277647 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/171f57a0-37f5-4b66-b8c7-dea91ddd1437-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wpnfr\" (UID: \"171f57a0-37f5-4b66-b8c7-dea91ddd1437\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:21.277696 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.277666 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/171f57a0-37f5-4b66-b8c7-dea91ddd1437-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wpnfr\" (UID: \"171f57a0-37f5-4b66-b8c7-dea91ddd1437\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:21.279511 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.279481 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/171f57a0-37f5-4b66-b8c7-dea91ddd1437-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wpnfr\" (UID: \"171f57a0-37f5-4b66-b8c7-dea91ddd1437\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:21.279769 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.279752 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/171f57a0-37f5-4b66-b8c7-dea91ddd1437-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wpnfr\" (UID: \"171f57a0-37f5-4b66-b8c7-dea91ddd1437\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:21.284198 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.284178 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tvcv\" (UniqueName: \"kubernetes.io/projected/171f57a0-37f5-4b66-b8c7-dea91ddd1437-kube-api-access-5tvcv\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wpnfr\" (UID: \"171f57a0-37f5-4b66-b8c7-dea91ddd1437\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:21.413563 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.413488 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:21.543825 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.543796 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr"] Jul 29 14:46:21.547038 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:46:21.547000 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod171f57a0_37f5_4b66_b8c7_dea91ddd1437.slice/crio-a8f0b381910096f6fec00dafb32b4a0e74580f2be1128cbe4393adfd222f714c WatchSource:0}: Error finding container a8f0b381910096f6fec00dafb32b4a0e74580f2be1128cbe4393adfd222f714c: Status 404 returned error can't find the container with id a8f0b381910096f6fec00dafb32b4a0e74580f2be1128cbe4393adfd222f714c Jul 29 14:46:21.622820 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.622796 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:21.622963 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.622854 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:21.635114 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.635094 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:21.984697 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.984663 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" event={"ID":"171f57a0-37f5-4b66-b8c7-dea91ddd1437","Type":"ContainerStarted","Data":"a8f0b381910096f6fec00dafb32b4a0e74580f2be1128cbe4393adfd222f714c"} Jul 29 14:46:21.995448 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:21.995423 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2qzhd" Jul 29 14:46:22.990836 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:22.990797 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" event={"ID":"171f57a0-37f5-4b66-b8c7-dea91ddd1437","Type":"ContainerStarted","Data":"1e38eeb8007bd0db21007228ec23444aef3af36f583b96521e3eeef6ae2a2cd9"} Jul 29 14:46:23.008188 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:23.008146 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" podStartSLOduration=1.516640143 podStartE2EDuration="2.00813005s" totalImagesPullingTime="491.489907ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:46:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:46:23.00628411 +0000 UTC m=+659.734433334" watchObservedRunningTime="2026-07-29 14:46:23.00813005 +0000 UTC m=+659.736279238" Jul 29 14:46:31.414625 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:31.414598 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:31.415082 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:31.414648 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:31.426559 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:31.426535 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:32.047711 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:32.047683 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wpnfr" Jul 29 14:46:37.334020 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:37.333981 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:46:51.120800 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:51.120767 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:46:53.533393 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:46:53.533361 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:47:39.238562 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:47:39.238530 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:47:39.389762 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:47:39.389701 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:47:40.334028 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:47:40.333989 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:47:40.479830 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:47:40.479798 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:48:13.568864 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:48:13.568831 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:49:44.977306 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:49:44.977269 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:49:45.264090 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:49:45.264053 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:49:46.173193 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:49:46.173159 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:50:23.736890 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:50:23.736858 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 14:50:23.739613 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:50:23.739590 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 14:50:23.780844 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:50:23.780824 2580 scope.go:117] "RemoveContainer" containerID="76d62369b4d86f456f229a55153da797ec513b7db09b975b1b4852fd6f1e9da8" Jul 29 14:50:27.934626 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:50:27.934588 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:50:28.032303 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:50:28.032269 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:50:28.148135 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:50:28.148103 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:50:34.297222 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:50:34.297189 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:50:34.398388 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:50:34.398355 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:50:34.531358 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:50:34.531332 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:50:40.333138 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:50:40.333096 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:50:42.498107 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:50:42.498069 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:51:27.203035 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:51:27.202983 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:52:27.676740 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:52:27.676679 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:52:28.190006 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:52:28.189972 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:53:11.659564 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:53:11.659523 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:53:11.760043 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:53:11.760016 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:54:39.854713 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:54:39.854677 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:54:45.963489 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:54:45.963398 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg"] Jul 29 14:54:45.963960 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:54:45.963670 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg" podUID="1914971c-3548-4f9f-8c83-80b25eb4f188" containerName="manager" containerID="cri-o://c0b24ab01cf58ecaa9c2b7fa6041aff24161e097a26ddaa5f6d4bf440447dc91" gracePeriod=10 Jul 29 14:54:49.213737 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:54:49.213694 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg" Jul 29 14:54:49.284396 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:54:49.284370 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1914971c-3548-4f9f-8c83-80b25eb4f188-extensions-socket-volume\") pod \"1914971c-3548-4f9f-8c83-80b25eb4f188\" (UID: \"1914971c-3548-4f9f-8c83-80b25eb4f188\") " Jul 29 14:54:49.284511 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:54:49.284444 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ggl9\" (UniqueName: \"kubernetes.io/projected/1914971c-3548-4f9f-8c83-80b25eb4f188-kube-api-access-6ggl9\") pod \"1914971c-3548-4f9f-8c83-80b25eb4f188\" (UID: \"1914971c-3548-4f9f-8c83-80b25eb4f188\") " Jul 29 14:54:49.284764 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:54:49.284737 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1914971c-3548-4f9f-8c83-80b25eb4f188-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1914971c-3548-4f9f-8c83-80b25eb4f188" (UID: "1914971c-3548-4f9f-8c83-80b25eb4f188"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 14:54:49.286270 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:54:49.286254 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1914971c-3548-4f9f-8c83-80b25eb4f188-kube-api-access-6ggl9" (OuterVolumeSpecName: "kube-api-access-6ggl9") pod "1914971c-3548-4f9f-8c83-80b25eb4f188" (UID: "1914971c-3548-4f9f-8c83-80b25eb4f188"). InnerVolumeSpecName "kube-api-access-6ggl9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 14:54:49.385349 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:54:49.385287 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ggl9\" (UniqueName: \"kubernetes.io/projected/1914971c-3548-4f9f-8c83-80b25eb4f188-kube-api-access-6ggl9\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:54:49.385349 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:54:49.385310 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1914971c-3548-4f9f-8c83-80b25eb4f188-extensions-socket-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 14:54:49.602196 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:54:49.602168 2580 generic.go:356] "Generic (PLEG): container finished" podID="1914971c-3548-4f9f-8c83-80b25eb4f188" containerID="c0b24ab01cf58ecaa9c2b7fa6041aff24161e097a26ddaa5f6d4bf440447dc91" exitCode=0 Jul 29 14:54:49.602319 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:54:49.602218 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg" Jul 29 14:54:49.602319 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:54:49.602230 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg" event={"ID":"1914971c-3548-4f9f-8c83-80b25eb4f188","Type":"ContainerDied","Data":"c0b24ab01cf58ecaa9c2b7fa6041aff24161e097a26ddaa5f6d4bf440447dc91"} Jul 29 14:54:49.602319 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:54:49.602273 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg" event={"ID":"1914971c-3548-4f9f-8c83-80b25eb4f188","Type":"ContainerDied","Data":"12e41fae0c316f36a8aa4e7d0b10978126dfb2b17061725c2b1fe8131a85c4b3"} Jul 29 14:54:49.602319 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:54:49.602296 2580 scope.go:117] "RemoveContainer" containerID="c0b24ab01cf58ecaa9c2b7fa6041aff24161e097a26ddaa5f6d4bf440447dc91" Jul 29 14:54:49.613740 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:54:49.613712 2580 scope.go:117] "RemoveContainer" containerID="c0b24ab01cf58ecaa9c2b7fa6041aff24161e097a26ddaa5f6d4bf440447dc91" Jul 29 14:54:49.614019 ip-10-0-132-42 kubenswrapper[2580]: E0729 14:54:49.613999 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0b24ab01cf58ecaa9c2b7fa6041aff24161e097a26ddaa5f6d4bf440447dc91\": container with ID starting with c0b24ab01cf58ecaa9c2b7fa6041aff24161e097a26ddaa5f6d4bf440447dc91 not found: ID does not exist" containerID="c0b24ab01cf58ecaa9c2b7fa6041aff24161e097a26ddaa5f6d4bf440447dc91" Jul 29 14:54:49.614147 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:54:49.614026 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0b24ab01cf58ecaa9c2b7fa6041aff24161e097a26ddaa5f6d4bf440447dc91"} err="failed to get container status \"c0b24ab01cf58ecaa9c2b7fa6041aff24161e097a26ddaa5f6d4bf440447dc91\": rpc error: code = NotFound desc = could not find container \"c0b24ab01cf58ecaa9c2b7fa6041aff24161e097a26ddaa5f6d4bf440447dc91\": container with ID starting with c0b24ab01cf58ecaa9c2b7fa6041aff24161e097a26ddaa5f6d4bf440447dc91 not found: ID does not exist" Jul 29 14:54:49.623412 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:54:49.623388 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg"] Jul 29 14:54:49.629381 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:54:49.629357 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-62cwg"] Jul 29 14:54:49.817636 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:54:49.817607 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1914971c-3548-4f9f-8c83-80b25eb4f188" path="/var/lib/kubelet/pods/1914971c-3548-4f9f-8c83-80b25eb4f188/volumes" Jul 29 14:55:23.779644 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:23.779610 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 14:55:23.780347 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:23.780330 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 14:55:47.179286 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:47.179249 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwlph"] Jul 29 14:55:47.181697 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:47.179635 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1914971c-3548-4f9f-8c83-80b25eb4f188" containerName="manager" Jul 29 14:55:47.181697 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:47.179646 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1914971c-3548-4f9f-8c83-80b25eb4f188" containerName="manager" Jul 29 14:55:47.181697 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:47.179708 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1914971c-3548-4f9f-8c83-80b25eb4f188" containerName="manager" Jul 29 14:55:47.182604 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:47.182587 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwlph" Jul 29 14:55:47.186466 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:47.186439 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rhxm9\"" Jul 29 14:55:47.198172 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:47.198142 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwlph"] Jul 29 14:55:47.219247 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:47.219218 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6790323f-ca7d-40af-8552-8ca4d350078a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kwlph\" (UID: \"6790323f-ca7d-40af-8552-8ca4d350078a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwlph" Jul 29 14:55:47.219346 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:47.219275 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qtdz\" (UniqueName: \"kubernetes.io/projected/6790323f-ca7d-40af-8552-8ca4d350078a-kube-api-access-7qtdz\") pod \"kuadrant-operator-controller-manager-86878c769f-kwlph\" (UID: \"6790323f-ca7d-40af-8552-8ca4d350078a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwlph" Jul 29 14:55:47.320312 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:47.320285 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6790323f-ca7d-40af-8552-8ca4d350078a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kwlph\" (UID: \"6790323f-ca7d-40af-8552-8ca4d350078a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwlph" Jul 29 14:55:47.320456 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:47.320334 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qtdz\" (UniqueName: \"kubernetes.io/projected/6790323f-ca7d-40af-8552-8ca4d350078a-kube-api-access-7qtdz\") pod \"kuadrant-operator-controller-manager-86878c769f-kwlph\" (UID: \"6790323f-ca7d-40af-8552-8ca4d350078a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwlph" Jul 29 14:55:47.320697 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:47.320670 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6790323f-ca7d-40af-8552-8ca4d350078a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kwlph\" (UID: \"6790323f-ca7d-40af-8552-8ca4d350078a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwlph" Jul 29 14:55:47.331880 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:47.331853 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qtdz\" (UniqueName: \"kubernetes.io/projected/6790323f-ca7d-40af-8552-8ca4d350078a-kube-api-access-7qtdz\") pod \"kuadrant-operator-controller-manager-86878c769f-kwlph\" (UID: \"6790323f-ca7d-40af-8552-8ca4d350078a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwlph" Jul 29 14:55:47.492602 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:47.492547 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwlph" Jul 29 14:55:47.638936 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:47.638911 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwlph"] Jul 29 14:55:47.640524 ip-10-0-132-42 kubenswrapper[2580]: W0729 14:55:47.640498 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6790323f_ca7d_40af_8552_8ca4d350078a.slice/crio-e8cacbb2b168a6056bad79ea49a4a3be1d67aa04d458fccb941860080b858e2c WatchSource:0}: Error finding container e8cacbb2b168a6056bad79ea49a4a3be1d67aa04d458fccb941860080b858e2c: Status 404 returned error can't find the container with id e8cacbb2b168a6056bad79ea49a4a3be1d67aa04d458fccb941860080b858e2c Jul 29 14:55:47.928434 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:47.928389 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwlph" event={"ID":"6790323f-ca7d-40af-8552-8ca4d350078a","Type":"ContainerStarted","Data":"76d248643bc38d5e2529823274c09110bfb2740041a91d4fd416ffbab65d362f"} Jul 29 14:55:47.928591 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:47.928470 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwlph" Jul 29 14:55:47.928591 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:47.928494 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwlph" event={"ID":"6790323f-ca7d-40af-8552-8ca4d350078a","Type":"ContainerStarted","Data":"e8cacbb2b168a6056bad79ea49a4a3be1d67aa04d458fccb941860080b858e2c"} Jul 29 14:55:47.986437 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:47.983389 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwlph" podStartSLOduration=0.983370283 podStartE2EDuration="983.370283ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 14:55:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 14:55:47.980221219 +0000 UTC m=+1224.708370461" watchObservedRunningTime="2026-07-29 14:55:47.983370283 +0000 UTC m=+1224.711519472" Jul 29 14:55:58.937286 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:55:58.937246 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kwlph" Jul 29 14:57:30.479029 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:57:30.478994 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:57:32.046158 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:57:32.046114 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:58:29.245386 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:58:29.245343 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:58:45.338579 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:58:45.338544 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:59:28.725112 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:59:28.725069 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 14:59:28.825647 ip-10-0-132-42 kubenswrapper[2580]: I0729 14:59:28.825617 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:00:00.133158 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:00:00.133124 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755620-t56f7"] Jul 29 15:00:00.136089 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:00:00.136071 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755620-t56f7" Jul 29 15:00:00.138280 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:00:00.138257 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mmt6p\"" Jul 29 15:00:00.143942 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:00:00.143312 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755620-t56f7"] Jul 29 15:00:00.146641 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:00:00.146620 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755620-t56f7" Jul 29 15:00:00.276234 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:00:00.276207 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755620-t56f7"] Jul 29 15:00:00.278016 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:00:00.277986 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod868e3483_c702_4028_ba6d_daf8254d4eb1.slice/crio-76fab079c58ce795c3520cfbefd293625561cd9a65cc9e010eeb20d5fea74b79 WatchSource:0}: Error finding container 76fab079c58ce795c3520cfbefd293625561cd9a65cc9e010eeb20d5fea74b79: Status 404 returned error can't find the container with id 76fab079c58ce795c3520cfbefd293625561cd9a65cc9e010eeb20d5fea74b79 Jul 29 15:00:00.279788 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:00:00.279751 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 15:00:01.248814 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:00:01.248777 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755620-t56f7" event={"ID":"868e3483-c702-4028-ba6d-daf8254d4eb1","Type":"ContainerStarted","Data":"76fab079c58ce795c3520cfbefd293625561cd9a65cc9e010eeb20d5fea74b79"} Jul 29 15:00:03.262798 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:00:03.262763 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755620-t56f7" event={"ID":"868e3483-c702-4028-ba6d-daf8254d4eb1","Type":"ContainerStarted","Data":"57dc3f092a3168107226e5ddf3d533f7cb1bc08ec6fe753a1edf82ac0c04bc9c"} Jul 29 15:00:03.279019 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:00:03.278974 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755620-t56f7" podStartSLOduration=1.073148076 podStartE2EDuration="3.278958839s" totalImagesPullingTime="2.205810763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:00:03.276476412 +0000 UTC m=+1480.004625637" watchObservedRunningTime="2026-07-29 15:00:03.278958839 +0000 UTC m=+1480.007108028" Jul 29 15:00:04.269368 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:00:04.269330 2580 generic.go:356] "Generic (PLEG): container finished" podID="868e3483-c702-4028-ba6d-daf8254d4eb1" containerID="57dc3f092a3168107226e5ddf3d533f7cb1bc08ec6fe753a1edf82ac0c04bc9c" exitCode=0 Jul 29 15:00:04.269777 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:00:04.269396 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755620-t56f7" event={"ID":"868e3483-c702-4028-ba6d-daf8254d4eb1","Type":"ContainerDied","Data":"57dc3f092a3168107226e5ddf3d533f7cb1bc08ec6fe753a1edf82ac0c04bc9c"} Jul 29 15:00:05.415676 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:00:05.415653 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755620-t56f7" Jul 29 15:00:06.287814 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:00:06.287786 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755620-t56f7" Jul 29 15:00:06.287814 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:00:06.287804 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755620-t56f7" event={"ID":"868e3483-c702-4028-ba6d-daf8254d4eb1","Type":"ContainerDied","Data":"76fab079c58ce795c3520cfbefd293625561cd9a65cc9e010eeb20d5fea74b79"} Jul 29 15:00:06.288010 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:00:06.287852 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="76fab079c58ce795c3520cfbefd293625561cd9a65cc9e010eeb20d5fea74b79" Jul 29 15:00:23.816842 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:00:23.816813 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 15:00:23.818619 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:00:23.818595 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 15:00:57.222094 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:00:57.222052 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:01:57.548796 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:01:57.548755 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:27.472931 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:02:27.472887 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:49.067366 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:02:49.067329 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:52.159341 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:02:52.159299 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:52.265437 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:02:52.265399 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:52.600684 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:02:52.600651 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:58.440380 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:02:58.440343 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:58.540637 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:02:58.540588 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:58.687212 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:02:58.687169 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:59.064528 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:02:59.064485 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:02:59.369081 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:02:59.368983 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:01.865299 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:01.865255 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:01.950537 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:01.950460 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:02.021585 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.021554 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:02.309830 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.309792 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:02.658180 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.658085 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:02.757561 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.757528 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:02.765128 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.765100 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557"] Jul 29 15:03:02.765485 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.765472 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="868e3483-c702-4028-ba6d-daf8254d4eb1" containerName="cleanup" Jul 29 15:03:02.765531 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.765492 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="868e3483-c702-4028-ba6d-daf8254d4eb1" containerName="cleanup" Jul 29 15:03:02.765595 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.765586 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="868e3483-c702-4028-ba6d-daf8254d4eb1" containerName="cleanup" Jul 29 15:03:02.768841 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.768822 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557" Jul 29 15:03:02.770778 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.770755 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-59092fc4\"" Jul 29 15:03:02.770869 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.770829 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-59092fc4-dockercfg-zcjms\"" Jul 29 15:03:02.779950 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.779927 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557"] Jul 29 15:03:02.806100 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.806076 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg"] Jul 29 15:03:02.810312 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.810287 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg" Jul 29 15:03:02.818036 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.818015 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg"] Jul 29 15:03:02.858047 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.858018 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwg24\" (UniqueName: \"kubernetes.io/projected/c54a06f2-37f4-4446-b030-298a5f212d3f-kube-api-access-dwg24\") pod \"payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557\" (UID: \"c54a06f2-37f4-4446-b030-298a5f212d3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557" Jul 29 15:03:02.858156 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.858101 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c54a06f2-37f4-4446-b030-298a5f212d3f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557\" (UID: \"c54a06f2-37f4-4446-b030-298a5f212d3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557" Jul 29 15:03:02.958936 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.958847 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqttm\" (UniqueName: \"kubernetes.io/projected/c985883f-b8f9-487c-9614-ae7797b326db-kube-api-access-qqttm\") pod \"payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg\" (UID: \"c985883f-b8f9-487c-9614-ae7797b326db\") " pod="openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg" Jul 29 15:03:02.958936 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.958895 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwg24\" (UniqueName: \"kubernetes.io/projected/c54a06f2-37f4-4446-b030-298a5f212d3f-kube-api-access-dwg24\") pod \"payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557\" (UID: \"c54a06f2-37f4-4446-b030-298a5f212d3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557" Jul 29 15:03:02.959345 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.958976 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c54a06f2-37f4-4446-b030-298a5f212d3f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557\" (UID: \"c54a06f2-37f4-4446-b030-298a5f212d3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557" Jul 29 15:03:02.959345 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.959005 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c985883f-b8f9-487c-9614-ae7797b326db-plugins-config-volume\") pod \"payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg\" (UID: \"c985883f-b8f9-487c-9614-ae7797b326db\") " pod="openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg" Jul 29 15:03:02.959892 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.959873 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c54a06f2-37f4-4446-b030-298a5f212d3f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557\" (UID: \"c54a06f2-37f4-4446-b030-298a5f212d3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557" Jul 29 15:03:02.968940 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:02.968916 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwg24\" (UniqueName: \"kubernetes.io/projected/c54a06f2-37f4-4446-b030-298a5f212d3f-kube-api-access-dwg24\") pod \"payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557\" (UID: \"c54a06f2-37f4-4446-b030-298a5f212d3f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557" Jul 29 15:03:03.059925 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:03.059888 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqttm\" (UniqueName: \"kubernetes.io/projected/c985883f-b8f9-487c-9614-ae7797b326db-kube-api-access-qqttm\") pod \"payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg\" (UID: \"c985883f-b8f9-487c-9614-ae7797b326db\") " pod="openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg" Jul 29 15:03:03.060102 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:03.059978 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c985883f-b8f9-487c-9614-ae7797b326db-plugins-config-volume\") pod \"payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg\" (UID: \"c985883f-b8f9-487c-9614-ae7797b326db\") " pod="openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg" Jul 29 15:03:03.060787 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:03.060768 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c985883f-b8f9-487c-9614-ae7797b326db-plugins-config-volume\") pod \"payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg\" (UID: \"c985883f-b8f9-487c-9614-ae7797b326db\") " pod="openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg" Jul 29 15:03:03.067581 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:03.067557 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqttm\" (UniqueName: \"kubernetes.io/projected/c985883f-b8f9-487c-9614-ae7797b326db-kube-api-access-qqttm\") pod \"payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg\" (UID: \"c985883f-b8f9-487c-9614-ae7797b326db\") " pod="openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg" Jul 29 15:03:03.079552 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:03.079528 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557" Jul 29 15:03:03.124573 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:03.124538 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg" Jul 29 15:03:03.245520 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:03.245492 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557"] Jul 29 15:03:03.249373 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:03:03.249327 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc54a06f2_37f4_4446_b030_298a5f212d3f.slice/crio-e5e6288c3c2bfa4ed150e38dc01b893a69fb74677c9a557e62ced4c2fd02cabe WatchSource:0}: Error finding container e5e6288c3c2bfa4ed150e38dc01b893a69fb74677c9a557e62ced4c2fd02cabe: Status 404 returned error can't find the container with id e5e6288c3c2bfa4ed150e38dc01b893a69fb74677c9a557e62ced4c2fd02cabe Jul 29 15:03:03.287222 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:03.287202 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg"] Jul 29 15:03:03.294930 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:03:03.294899 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc985883f_b8f9_487c_9614_ae7797b326db.slice/crio-ded5f617237819a0dc093b3f71c2d1af5fc45853c0cbf2cc5d0438ca4fc8d46b WatchSource:0}: Error finding container ded5f617237819a0dc093b3f71c2d1af5fc45853c0cbf2cc5d0438ca4fc8d46b: Status 404 returned error can't find the container with id ded5f617237819a0dc093b3f71c2d1af5fc45853c0cbf2cc5d0438ca4fc8d46b Jul 29 15:03:04.220252 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:04.220216 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg" event={"ID":"c985883f-b8f9-487c-9614-ae7797b326db","Type":"ContainerStarted","Data":"3f662ac1d17b995617f99b18d2b071cb1919ddacd958c895551dbfc0194512bb"} Jul 29 15:03:04.220709 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:04.220274 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg" event={"ID":"c985883f-b8f9-487c-9614-ae7797b326db","Type":"ContainerStarted","Data":"ded5f617237819a0dc093b3f71c2d1af5fc45853c0cbf2cc5d0438ca4fc8d46b"} Jul 29 15:03:04.220709 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:04.220388 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg" Jul 29 15:03:04.221700 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:04.221675 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557" event={"ID":"c54a06f2-37f4-4446-b030-298a5f212d3f","Type":"ContainerStarted","Data":"df6672c94e671d129ea11d6b8f2a500a16228ab7e3325a8e6e339b00e743cc7d"} Jul 29 15:03:04.221819 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:04.221717 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557" event={"ID":"c54a06f2-37f4-4446-b030-298a5f212d3f","Type":"ContainerStarted","Data":"e5e6288c3c2bfa4ed150e38dc01b893a69fb74677c9a557e62ced4c2fd02cabe"} Jul 29 15:03:04.221819 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:04.221771 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557" Jul 29 15:03:04.238064 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:04.238026 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg" podStartSLOduration=2.238013004 podStartE2EDuration="2.238013004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:03:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:03:04.236408555 +0000 UTC m=+1660.964557761" watchObservedRunningTime="2026-07-29 15:03:04.238013004 +0000 UTC m=+1660.966162192" Jul 29 15:03:04.253332 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:04.253288 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557" podStartSLOduration=2.253273104 podStartE2EDuration="2.253273104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:03:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:03:04.252377509 +0000 UTC m=+1660.980526701" watchObservedRunningTime="2026-07-29 15:03:04.253273104 +0000 UTC m=+1660.981422291" Jul 29 15:03:15.229369 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:15.229336 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg" Jul 29 15:03:15.229836 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:15.229406 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557" Jul 29 15:03:42.607297 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:42.607260 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:42.636023 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:42.635989 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg"] Jul 29 15:03:42.636265 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:42.636240 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg" podUID="c985883f-b8f9-487c-9614-ae7797b326db" containerName="payload-processing" containerID="cri-o://3f662ac1d17b995617f99b18d2b071cb1919ddacd958c895551dbfc0194512bb" gracePeriod=30 Jul 29 15:03:42.646404 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:42.646377 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557"] Jul 29 15:03:42.646629 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:42.646604 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557" podUID="c54a06f2-37f4-4446-b030-298a5f212d3f" containerName="payload-pre-processing" containerID="cri-o://df6672c94e671d129ea11d6b8f2a500a16228ab7e3325a8e6e339b00e743cc7d" gracePeriod=30 Jul 29 15:03:42.704946 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:42.704902 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:42.803863 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:42.803823 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:42.878115 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:42.878049 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:42.898995 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:42.898970 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg" Jul 29 15:03:42.933211 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:42.933182 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557" Jul 29 15:03:42.989973 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:42.989934 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:43.001959 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.001933 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c54a06f2-37f4-4446-b030-298a5f212d3f-plugins-config-volume\") pod \"c54a06f2-37f4-4446-b030-298a5f212d3f\" (UID: \"c54a06f2-37f4-4446-b030-298a5f212d3f\") " Jul 29 15:03:43.002090 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.001979 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c985883f-b8f9-487c-9614-ae7797b326db-plugins-config-volume\") pod \"c985883f-b8f9-487c-9614-ae7797b326db\" (UID: \"c985883f-b8f9-487c-9614-ae7797b326db\") " Jul 29 15:03:43.002156 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.002131 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwg24\" (UniqueName: \"kubernetes.io/projected/c54a06f2-37f4-4446-b030-298a5f212d3f-kube-api-access-dwg24\") pod \"c54a06f2-37f4-4446-b030-298a5f212d3f\" (UID: \"c54a06f2-37f4-4446-b030-298a5f212d3f\") " Jul 29 15:03:43.002213 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.002167 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqttm\" (UniqueName: \"kubernetes.io/projected/c985883f-b8f9-487c-9614-ae7797b326db-kube-api-access-qqttm\") pod \"c985883f-b8f9-487c-9614-ae7797b326db\" (UID: \"c985883f-b8f9-487c-9614-ae7797b326db\") " Jul 29 15:03:43.002299 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.002277 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c985883f-b8f9-487c-9614-ae7797b326db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c985883f-b8f9-487c-9614-ae7797b326db" (UID: "c985883f-b8f9-487c-9614-ae7797b326db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:03:43.002363 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.002331 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c54a06f2-37f4-4446-b030-298a5f212d3f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c54a06f2-37f4-4446-b030-298a5f212d3f" (UID: "c54a06f2-37f4-4446-b030-298a5f212d3f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:03:43.002539 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.002523 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c54a06f2-37f4-4446-b030-298a5f212d3f-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:03:43.002600 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.002542 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c985883f-b8f9-487c-9614-ae7797b326db-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:03:43.004208 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.004190 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c54a06f2-37f4-4446-b030-298a5f212d3f-kube-api-access-dwg24" (OuterVolumeSpecName: "kube-api-access-dwg24") pod "c54a06f2-37f4-4446-b030-298a5f212d3f" (UID: "c54a06f2-37f4-4446-b030-298a5f212d3f"). InnerVolumeSpecName "kube-api-access-dwg24". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:03:43.004276 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.004258 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c985883f-b8f9-487c-9614-ae7797b326db-kube-api-access-qqttm" (OuterVolumeSpecName: "kube-api-access-qqttm") pod "c985883f-b8f9-487c-9614-ae7797b326db" (UID: "c985883f-b8f9-487c-9614-ae7797b326db"). InnerVolumeSpecName "kube-api-access-qqttm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:03:43.103356 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.103328 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwg24\" (UniqueName: \"kubernetes.io/projected/c54a06f2-37f4-4446-b030-298a5f212d3f-kube-api-access-dwg24\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:03:43.103356 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.103354 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqttm\" (UniqueName: \"kubernetes.io/projected/c985883f-b8f9-487c-9614-ae7797b326db-kube-api-access-qqttm\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:03:43.236471 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.236388 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:43.431973 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.431932 2580 generic.go:356] "Generic (PLEG): container finished" podID="c985883f-b8f9-487c-9614-ae7797b326db" containerID="3f662ac1d17b995617f99b18d2b071cb1919ddacd958c895551dbfc0194512bb" exitCode=0 Jul 29 15:03:43.432158 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.431996 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg" Jul 29 15:03:43.432158 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.432020 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg" event={"ID":"c985883f-b8f9-487c-9614-ae7797b326db","Type":"ContainerDied","Data":"3f662ac1d17b995617f99b18d2b071cb1919ddacd958c895551dbfc0194512bb"} Jul 29 15:03:43.432158 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.432111 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg" event={"ID":"c985883f-b8f9-487c-9614-ae7797b326db","Type":"ContainerDied","Data":"ded5f617237819a0dc093b3f71c2d1af5fc45853c0cbf2cc5d0438ca4fc8d46b"} Jul 29 15:03:43.432158 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.432142 2580 scope.go:117] "RemoveContainer" containerID="3f662ac1d17b995617f99b18d2b071cb1919ddacd958c895551dbfc0194512bb" Jul 29 15:03:43.433453 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.433430 2580 generic.go:356] "Generic (PLEG): container finished" podID="c54a06f2-37f4-4446-b030-298a5f212d3f" containerID="df6672c94e671d129ea11d6b8f2a500a16228ab7e3325a8e6e339b00e743cc7d" exitCode=0 Jul 29 15:03:43.433659 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.433488 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557" Jul 29 15:03:43.433758 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.433513 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557" event={"ID":"c54a06f2-37f4-4446-b030-298a5f212d3f","Type":"ContainerDied","Data":"df6672c94e671d129ea11d6b8f2a500a16228ab7e3325a8e6e339b00e743cc7d"} Jul 29 15:03:43.433828 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.433786 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557" event={"ID":"c54a06f2-37f4-4446-b030-298a5f212d3f","Type":"ContainerDied","Data":"e5e6288c3c2bfa4ed150e38dc01b893a69fb74677c9a557e62ced4c2fd02cabe"} Jul 29 15:03:43.441480 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.441463 2580 scope.go:117] "RemoveContainer" containerID="3f662ac1d17b995617f99b18d2b071cb1919ddacd958c895551dbfc0194512bb" Jul 29 15:03:43.441865 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:03:43.441837 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f662ac1d17b995617f99b18d2b071cb1919ddacd958c895551dbfc0194512bb\": container with ID starting with 3f662ac1d17b995617f99b18d2b071cb1919ddacd958c895551dbfc0194512bb not found: ID does not exist" containerID="3f662ac1d17b995617f99b18d2b071cb1919ddacd958c895551dbfc0194512bb" Jul 29 15:03:43.441975 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.441876 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f662ac1d17b995617f99b18d2b071cb1919ddacd958c895551dbfc0194512bb"} err="failed to get container status \"3f662ac1d17b995617f99b18d2b071cb1919ddacd958c895551dbfc0194512bb\": rpc error: code = NotFound desc = could not find container \"3f662ac1d17b995617f99b18d2b071cb1919ddacd958c895551dbfc0194512bb\": container with ID starting with 3f662ac1d17b995617f99b18d2b071cb1919ddacd958c895551dbfc0194512bb not found: ID does not exist" Jul 29 15:03:43.441975 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.441899 2580 scope.go:117] "RemoveContainer" containerID="df6672c94e671d129ea11d6b8f2a500a16228ab7e3325a8e6e339b00e743cc7d" Jul 29 15:03:43.452097 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.452067 2580 scope.go:117] "RemoveContainer" containerID="df6672c94e671d129ea11d6b8f2a500a16228ab7e3325a8e6e339b00e743cc7d" Jul 29 15:03:43.452380 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:03:43.452355 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df6672c94e671d129ea11d6b8f2a500a16228ab7e3325a8e6e339b00e743cc7d\": container with ID starting with df6672c94e671d129ea11d6b8f2a500a16228ab7e3325a8e6e339b00e743cc7d not found: ID does not exist" containerID="df6672c94e671d129ea11d6b8f2a500a16228ab7e3325a8e6e339b00e743cc7d" Jul 29 15:03:43.452446 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.452392 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df6672c94e671d129ea11d6b8f2a500a16228ab7e3325a8e6e339b00e743cc7d"} err="failed to get container status \"df6672c94e671d129ea11d6b8f2a500a16228ab7e3325a8e6e339b00e743cc7d\": rpc error: code = NotFound desc = could not find container \"df6672c94e671d129ea11d6b8f2a500a16228ab7e3325a8e6e339b00e743cc7d\": container with ID starting with df6672c94e671d129ea11d6b8f2a500a16228ab7e3325a8e6e339b00e743cc7d not found: ID does not exist" Jul 29 15:03:43.464250 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.464225 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:43.470076 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.470055 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557"] Jul 29 15:03:43.479894 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.479861 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-59092fc4-6689556844-vt557"] Jul 29 15:03:43.497310 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.497290 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg"] Jul 29 15:03:43.508256 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.508233 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-59092fc4-7f7f57cd4-hxnpg"] Jul 29 15:03:43.565802 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.565772 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:43.715220 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.715177 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:43.818946 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.818871 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c54a06f2-37f4-4446-b030-298a5f212d3f" path="/var/lib/kubelet/pods/c54a06f2-37f4-4446-b030-298a5f212d3f/volumes" Jul 29 15:03:43.819268 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:43.819249 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c985883f-b8f9-487c-9614-ae7797b326db" path="/var/lib/kubelet/pods/c985883f-b8f9-487c-9614-ae7797b326db/volumes" Jul 29 15:03:44.074153 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:44.074066 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:44.630110 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:44.630075 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:46.274459 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:46.274379 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:46.397233 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:46.397190 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:46.568936 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:46.568845 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:47.128769 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.128692 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:47.295407 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.295367 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:03:47.326680 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.326647 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj"] Jul 29 15:03:47.327069 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.327054 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c985883f-b8f9-487c-9614-ae7797b326db" containerName="payload-processing" Jul 29 15:03:47.327110 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.327071 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c985883f-b8f9-487c-9614-ae7797b326db" containerName="payload-processing" Jul 29 15:03:47.327110 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.327089 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54a06f2-37f4-4446-b030-298a5f212d3f" containerName="payload-pre-processing" Jul 29 15:03:47.327110 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.327096 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54a06f2-37f4-4446-b030-298a5f212d3f" containerName="payload-pre-processing" Jul 29 15:03:47.327207 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.327153 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c54a06f2-37f4-4446-b030-298a5f212d3f" containerName="payload-pre-processing" Jul 29 15:03:47.327207 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.327164 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c985883f-b8f9-487c-9614-ae7797b326db" containerName="payload-processing" Jul 29 15:03:47.330494 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.330475 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj" Jul 29 15:03:47.335309 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.335276 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-079ae78c\"" Jul 29 15:03:47.335453 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.335365 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-079ae78c-dockercfg-tmn6p\"" Jul 29 15:03:47.348790 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.348756 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl"] Jul 29 15:03:47.352905 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.352880 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl" Jul 29 15:03:47.358253 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.358226 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj"] Jul 29 15:03:47.368452 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.368419 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl"] Jul 29 15:03:47.441407 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.441293 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88527\" (UniqueName: \"kubernetes.io/projected/0f8d9359-eca0-48ab-966f-e7c454a49166-kube-api-access-88527\") pod \"payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl\" (UID: \"0f8d9359-eca0-48ab-966f-e7c454a49166\") " pod="openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl" Jul 29 15:03:47.441570 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.441420 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8c22\" (UniqueName: \"kubernetes.io/projected/2528df94-43ae-4b1b-9e9e-25162171d5a1-kube-api-access-x8c22\") pod \"payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj\" (UID: \"2528df94-43ae-4b1b-9e9e-25162171d5a1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj" Jul 29 15:03:47.441570 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.441468 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2528df94-43ae-4b1b-9e9e-25162171d5a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj\" (UID: \"2528df94-43ae-4b1b-9e9e-25162171d5a1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj" Jul 29 15:03:47.441570 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.441495 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f8d9359-eca0-48ab-966f-e7c454a49166-plugins-config-volume\") pod \"payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl\" (UID: \"0f8d9359-eca0-48ab-966f-e7c454a49166\") " pod="openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl" Jul 29 15:03:47.542130 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.542097 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8c22\" (UniqueName: \"kubernetes.io/projected/2528df94-43ae-4b1b-9e9e-25162171d5a1-kube-api-access-x8c22\") pod \"payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj\" (UID: \"2528df94-43ae-4b1b-9e9e-25162171d5a1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj" Jul 29 15:03:47.542130 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.542143 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2528df94-43ae-4b1b-9e9e-25162171d5a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj\" (UID: \"2528df94-43ae-4b1b-9e9e-25162171d5a1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj" Jul 29 15:03:47.542418 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.542167 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f8d9359-eca0-48ab-966f-e7c454a49166-plugins-config-volume\") pod \"payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl\" (UID: \"0f8d9359-eca0-48ab-966f-e7c454a49166\") " pod="openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl" Jul 29 15:03:47.542418 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.542207 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88527\" (UniqueName: \"kubernetes.io/projected/0f8d9359-eca0-48ab-966f-e7c454a49166-kube-api-access-88527\") pod \"payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl\" (UID: \"0f8d9359-eca0-48ab-966f-e7c454a49166\") " pod="openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl" Jul 29 15:03:47.542886 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.542867 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2528df94-43ae-4b1b-9e9e-25162171d5a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj\" (UID: \"2528df94-43ae-4b1b-9e9e-25162171d5a1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj" Jul 29 15:03:47.542952 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.542881 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f8d9359-eca0-48ab-966f-e7c454a49166-plugins-config-volume\") pod \"payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl\" (UID: \"0f8d9359-eca0-48ab-966f-e7c454a49166\") " pod="openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl" Jul 29 15:03:47.552059 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.552029 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8c22\" (UniqueName: \"kubernetes.io/projected/2528df94-43ae-4b1b-9e9e-25162171d5a1-kube-api-access-x8c22\") pod \"payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj\" (UID: \"2528df94-43ae-4b1b-9e9e-25162171d5a1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj" Jul 29 15:03:47.552233 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.552170 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88527\" (UniqueName: \"kubernetes.io/projected/0f8d9359-eca0-48ab-966f-e7c454a49166-kube-api-access-88527\") pod \"payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl\" (UID: \"0f8d9359-eca0-48ab-966f-e7c454a49166\") " pod="openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl" Jul 29 15:03:47.640990 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.640960 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj" Jul 29 15:03:47.665186 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.665144 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl" Jul 29 15:03:47.798931 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.798890 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj"] Jul 29 15:03:47.800213 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:03:47.800183 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2528df94_43ae_4b1b_9e9e_25162171d5a1.slice/crio-1a89a1114a20ea34321c4c1a8391fd57f12f7f17a7ba331f1053524a0e32e6a4 WatchSource:0}: Error finding container 1a89a1114a20ea34321c4c1a8391fd57f12f7f17a7ba331f1053524a0e32e6a4: Status 404 returned error can't find the container with id 1a89a1114a20ea34321c4c1a8391fd57f12f7f17a7ba331f1053524a0e32e6a4 Jul 29 15:03:47.832637 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:47.832619 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl"] Jul 29 15:03:47.835241 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:03:47.835219 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f8d9359_eca0_48ab_966f_e7c454a49166.slice/crio-a486e9b04518f4c308fb8d45f7c14db8505827f3fa68475e429a5ccb6fa59aa0 WatchSource:0}: Error finding container a486e9b04518f4c308fb8d45f7c14db8505827f3fa68475e429a5ccb6fa59aa0: Status 404 returned error can't find the container with id a486e9b04518f4c308fb8d45f7c14db8505827f3fa68475e429a5ccb6fa59aa0 Jul 29 15:03:48.469174 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:48.469138 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj" event={"ID":"2528df94-43ae-4b1b-9e9e-25162171d5a1","Type":"ContainerStarted","Data":"a8251bd2d3fa319a9446bf6a092964f1fc47fbfdd29fc66b493a13e1d4483ef1"} Jul 29 15:03:48.469620 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:48.469193 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj" event={"ID":"2528df94-43ae-4b1b-9e9e-25162171d5a1","Type":"ContainerStarted","Data":"1a89a1114a20ea34321c4c1a8391fd57f12f7f17a7ba331f1053524a0e32e6a4"} Jul 29 15:03:48.469620 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:48.469224 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj" Jul 29 15:03:48.470806 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:48.470779 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl" event={"ID":"0f8d9359-eca0-48ab-966f-e7c454a49166","Type":"ContainerStarted","Data":"67a4047e5ce3a0cde9533fbec14a4ddc4fc88b94226d69e5bb8ff55113471900"} Jul 29 15:03:48.470937 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:48.470835 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl" Jul 29 15:03:48.470937 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:48.470859 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl" event={"ID":"0f8d9359-eca0-48ab-966f-e7c454a49166","Type":"ContainerStarted","Data":"a486e9b04518f4c308fb8d45f7c14db8505827f3fa68475e429a5ccb6fa59aa0"} Jul 29 15:03:48.487999 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:48.487925 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj" podStartSLOduration=1.4879025430000001 podStartE2EDuration="1.487902543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:03:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:03:48.485295606 +0000 UTC m=+1705.213444795" watchObservedRunningTime="2026-07-29 15:03:48.487902543 +0000 UTC m=+1705.216051785" Jul 29 15:03:48.505973 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:48.505918 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl" podStartSLOduration=1.505899248 podStartE2EDuration="1.505899248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:03:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:03:48.503340467 +0000 UTC m=+1705.231489656" watchObservedRunningTime="2026-07-29 15:03:48.505899248 +0000 UTC m=+1705.234048439" Jul 29 15:03:59.478459 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:59.478424 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj" Jul 29 15:03:59.478901 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:03:59.478777 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl" Jul 29 15:04:08.429209 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:08.429173 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:08.529932 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:08.529902 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:17.221493 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.221454 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:17.247987 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.247954 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl"] Jul 29 15:04:17.248220 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.248193 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl" podUID="0f8d9359-eca0-48ab-966f-e7c454a49166" containerName="payload-processing" containerID="cri-o://67a4047e5ce3a0cde9533fbec14a4ddc4fc88b94226d69e5bb8ff55113471900" gracePeriod=30 Jul 29 15:04:17.260016 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.259988 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj"] Jul 29 15:04:17.261134 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.260301 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj" podUID="2528df94-43ae-4b1b-9e9e-25162171d5a1" containerName="payload-pre-processing" containerID="cri-o://a8251bd2d3fa319a9446bf6a092964f1fc47fbfdd29fc66b493a13e1d4483ef1" gracePeriod=30 Jul 29 15:04:17.328190 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.328156 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:17.418744 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.413507 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:17.475632 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.475487 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:17.559119 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.559094 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl" Jul 29 15:04:17.562395 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.562376 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj" Jul 29 15:04:17.566347 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.566324 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:17.632251 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.632229 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8c22\" (UniqueName: \"kubernetes.io/projected/2528df94-43ae-4b1b-9e9e-25162171d5a1-kube-api-access-x8c22\") pod \"2528df94-43ae-4b1b-9e9e-25162171d5a1\" (UID: \"2528df94-43ae-4b1b-9e9e-25162171d5a1\") " Jul 29 15:04:17.632407 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.632276 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88527\" (UniqueName: \"kubernetes.io/projected/0f8d9359-eca0-48ab-966f-e7c454a49166-kube-api-access-88527\") pod \"0f8d9359-eca0-48ab-966f-e7c454a49166\" (UID: \"0f8d9359-eca0-48ab-966f-e7c454a49166\") " Jul 29 15:04:17.632407 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.632318 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f8d9359-eca0-48ab-966f-e7c454a49166-plugins-config-volume\") pod \"0f8d9359-eca0-48ab-966f-e7c454a49166\" (UID: \"0f8d9359-eca0-48ab-966f-e7c454a49166\") " Jul 29 15:04:17.632407 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.632347 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2528df94-43ae-4b1b-9e9e-25162171d5a1-plugins-config-volume\") pod \"2528df94-43ae-4b1b-9e9e-25162171d5a1\" (UID: \"2528df94-43ae-4b1b-9e9e-25162171d5a1\") " Jul 29 15:04:17.633041 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.632940 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f8d9359-eca0-48ab-966f-e7c454a49166-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0f8d9359-eca0-48ab-966f-e7c454a49166" (UID: "0f8d9359-eca0-48ab-966f-e7c454a49166"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:04:17.633041 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.632997 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2528df94-43ae-4b1b-9e9e-25162171d5a1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2528df94-43ae-4b1b-9e9e-25162171d5a1" (UID: "2528df94-43ae-4b1b-9e9e-25162171d5a1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:04:17.634763 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.634711 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f8d9359-eca0-48ab-966f-e7c454a49166-kube-api-access-88527" (OuterVolumeSpecName: "kube-api-access-88527") pod "0f8d9359-eca0-48ab-966f-e7c454a49166" (UID: "0f8d9359-eca0-48ab-966f-e7c454a49166"). InnerVolumeSpecName "kube-api-access-88527". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:04:17.634885 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.634836 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2528df94-43ae-4b1b-9e9e-25162171d5a1-kube-api-access-x8c22" (OuterVolumeSpecName: "kube-api-access-x8c22") pod "2528df94-43ae-4b1b-9e9e-25162171d5a1" (UID: "2528df94-43ae-4b1b-9e9e-25162171d5a1"). InnerVolumeSpecName "kube-api-access-x8c22". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:04:17.637581 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.637559 2580 generic.go:356] "Generic (PLEG): container finished" podID="2528df94-43ae-4b1b-9e9e-25162171d5a1" containerID="a8251bd2d3fa319a9446bf6a092964f1fc47fbfdd29fc66b493a13e1d4483ef1" exitCode=0 Jul 29 15:04:17.637681 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.637612 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj" Jul 29 15:04:17.637681 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.637639 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj" event={"ID":"2528df94-43ae-4b1b-9e9e-25162171d5a1","Type":"ContainerDied","Data":"a8251bd2d3fa319a9446bf6a092964f1fc47fbfdd29fc66b493a13e1d4483ef1"} Jul 29 15:04:17.637860 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.637741 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj" event={"ID":"2528df94-43ae-4b1b-9e9e-25162171d5a1","Type":"ContainerDied","Data":"1a89a1114a20ea34321c4c1a8391fd57f12f7f17a7ba331f1053524a0e32e6a4"} Jul 29 15:04:17.637860 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.637776 2580 scope.go:117] "RemoveContainer" containerID="a8251bd2d3fa319a9446bf6a092964f1fc47fbfdd29fc66b493a13e1d4483ef1" Jul 29 15:04:17.639037 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.639015 2580 generic.go:356] "Generic (PLEG): container finished" podID="0f8d9359-eca0-48ab-966f-e7c454a49166" containerID="67a4047e5ce3a0cde9533fbec14a4ddc4fc88b94226d69e5bb8ff55113471900" exitCode=0 Jul 29 15:04:17.639127 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.639046 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl" Jul 29 15:04:17.639127 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.639098 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl" event={"ID":"0f8d9359-eca0-48ab-966f-e7c454a49166","Type":"ContainerDied","Data":"67a4047e5ce3a0cde9533fbec14a4ddc4fc88b94226d69e5bb8ff55113471900"} Jul 29 15:04:17.639209 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.639151 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl" event={"ID":"0f8d9359-eca0-48ab-966f-e7c454a49166","Type":"ContainerDied","Data":"a486e9b04518f4c308fb8d45f7c14db8505827f3fa68475e429a5ccb6fa59aa0"} Jul 29 15:04:17.653609 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.653587 2580 scope.go:117] "RemoveContainer" containerID="a8251bd2d3fa319a9446bf6a092964f1fc47fbfdd29fc66b493a13e1d4483ef1" Jul 29 15:04:17.653948 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:04:17.653925 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8251bd2d3fa319a9446bf6a092964f1fc47fbfdd29fc66b493a13e1d4483ef1\": container with ID starting with a8251bd2d3fa319a9446bf6a092964f1fc47fbfdd29fc66b493a13e1d4483ef1 not found: ID does not exist" containerID="a8251bd2d3fa319a9446bf6a092964f1fc47fbfdd29fc66b493a13e1d4483ef1" Jul 29 15:04:17.654049 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.653952 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8251bd2d3fa319a9446bf6a092964f1fc47fbfdd29fc66b493a13e1d4483ef1"} err="failed to get container status \"a8251bd2d3fa319a9446bf6a092964f1fc47fbfdd29fc66b493a13e1d4483ef1\": rpc error: code = NotFound desc = could not find container \"a8251bd2d3fa319a9446bf6a092964f1fc47fbfdd29fc66b493a13e1d4483ef1\": container with ID starting with a8251bd2d3fa319a9446bf6a092964f1fc47fbfdd29fc66b493a13e1d4483ef1 not found: ID does not exist" Jul 29 15:04:17.654049 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.653970 2580 scope.go:117] "RemoveContainer" containerID="67a4047e5ce3a0cde9533fbec14a4ddc4fc88b94226d69e5bb8ff55113471900" Jul 29 15:04:17.662111 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.662095 2580 scope.go:117] "RemoveContainer" containerID="67a4047e5ce3a0cde9533fbec14a4ddc4fc88b94226d69e5bb8ff55113471900" Jul 29 15:04:17.662352 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:04:17.662335 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67a4047e5ce3a0cde9533fbec14a4ddc4fc88b94226d69e5bb8ff55113471900\": container with ID starting with 67a4047e5ce3a0cde9533fbec14a4ddc4fc88b94226d69e5bb8ff55113471900 not found: ID does not exist" containerID="67a4047e5ce3a0cde9533fbec14a4ddc4fc88b94226d69e5bb8ff55113471900" Jul 29 15:04:17.662416 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.662357 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67a4047e5ce3a0cde9533fbec14a4ddc4fc88b94226d69e5bb8ff55113471900"} err="failed to get container status \"67a4047e5ce3a0cde9533fbec14a4ddc4fc88b94226d69e5bb8ff55113471900\": rpc error: code = NotFound desc = could not find container \"67a4047e5ce3a0cde9533fbec14a4ddc4fc88b94226d69e5bb8ff55113471900\": container with ID starting with 67a4047e5ce3a0cde9533fbec14a4ddc4fc88b94226d69e5bb8ff55113471900 not found: ID does not exist" Jul 29 15:04:17.666477 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.666459 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj"] Jul 29 15:04:17.670835 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.670815 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-079ae78c-b9d7b5c48-nv9sj"] Jul 29 15:04:17.680800 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.680780 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl"] Jul 29 15:04:17.683927 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.683907 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-079ae78c-5cf59cfff7-5n8cl"] Jul 29 15:04:17.733989 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.733940 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8c22\" (UniqueName: \"kubernetes.io/projected/2528df94-43ae-4b1b-9e9e-25162171d5a1-kube-api-access-x8c22\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:04:17.733989 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.733958 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88527\" (UniqueName: \"kubernetes.io/projected/0f8d9359-eca0-48ab-966f-e7c454a49166-kube-api-access-88527\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:04:17.733989 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.733967 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f8d9359-eca0-48ab-966f-e7c454a49166-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:04:17.733989 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.733977 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2528df94-43ae-4b1b-9e9e-25162171d5a1-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:04:17.809157 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.809130 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:17.818293 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.818273 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f8d9359-eca0-48ab-966f-e7c454a49166" path="/var/lib/kubelet/pods/0f8d9359-eca0-48ab-966f-e7c454a49166/volumes" Jul 29 15:04:17.818624 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:17.818611 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2528df94-43ae-4b1b-9e9e-25162171d5a1" path="/var/lib/kubelet/pods/2528df94-43ae-4b1b-9e9e-25162171d5a1/volumes" Jul 29 15:04:18.041759 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:18.041708 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:18.354829 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:18.354701 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:18.452930 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:18.452887 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:20.259811 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:20.259771 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:20.348716 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:20.348680 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:20.418938 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:20.418908 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:20.629539 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:20.629454 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:20.929463 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:20.929361 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:21.022036 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.021988 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:21.040567 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.040534 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49"] Jul 29 15:04:21.040987 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.040972 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2528df94-43ae-4b1b-9e9e-25162171d5a1" containerName="payload-pre-processing" Jul 29 15:04:21.041032 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.040990 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2528df94-43ae-4b1b-9e9e-25162171d5a1" containerName="payload-pre-processing" Jul 29 15:04:21.041032 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.041005 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f8d9359-eca0-48ab-966f-e7c454a49166" containerName="payload-processing" Jul 29 15:04:21.041032 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.041013 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f8d9359-eca0-48ab-966f-e7c454a49166" containerName="payload-processing" Jul 29 15:04:21.041130 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.041082 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f8d9359-eca0-48ab-966f-e7c454a49166" containerName="payload-processing" Jul 29 15:04:21.041130 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.041093 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2528df94-43ae-4b1b-9e9e-25162171d5a1" containerName="payload-pre-processing" Jul 29 15:04:21.045679 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.045660 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb"] Jul 29 15:04:21.045806 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.045790 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49" Jul 29 15:04:21.049037 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.049019 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb" Jul 29 15:04:21.049410 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.049390 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-43963a57\"" Jul 29 15:04:21.049550 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.049535 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-43963a57-dockercfg-mfbhs\"" Jul 29 15:04:21.062515 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.062485 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb"] Jul 29 15:04:21.065153 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.065009 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49"] Jul 29 15:04:21.164675 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.164638 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7chzr\" (UniqueName: \"kubernetes.io/projected/7d62807e-8778-49f3-b1fb-a9d9286349da-kube-api-access-7chzr\") pod \"payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb\" (UID: \"7d62807e-8778-49f3-b1fb-a9d9286349da\") " pod="openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb" Jul 29 15:04:21.164675 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.164689 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8545c81-83aa-4467-a427-d8ce25e606c1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49\" (UID: \"b8545c81-83aa-4467-a427-d8ce25e606c1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49" Jul 29 15:04:21.164894 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.164795 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsnqk\" (UniqueName: \"kubernetes.io/projected/b8545c81-83aa-4467-a427-d8ce25e606c1-kube-api-access-wsnqk\") pod \"payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49\" (UID: \"b8545c81-83aa-4467-a427-d8ce25e606c1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49" Jul 29 15:04:21.164894 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.164844 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d62807e-8778-49f3-b1fb-a9d9286349da-plugins-config-volume\") pod \"payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb\" (UID: \"7d62807e-8778-49f3-b1fb-a9d9286349da\") " pod="openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb" Jul 29 15:04:21.265909 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.265871 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d62807e-8778-49f3-b1fb-a9d9286349da-plugins-config-volume\") pod \"payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb\" (UID: \"7d62807e-8778-49f3-b1fb-a9d9286349da\") " pod="openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb" Jul 29 15:04:21.266379 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.265929 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7chzr\" (UniqueName: \"kubernetes.io/projected/7d62807e-8778-49f3-b1fb-a9d9286349da-kube-api-access-7chzr\") pod \"payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb\" (UID: \"7d62807e-8778-49f3-b1fb-a9d9286349da\") " pod="openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb" Jul 29 15:04:21.266379 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.266056 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8545c81-83aa-4467-a427-d8ce25e606c1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49\" (UID: \"b8545c81-83aa-4467-a427-d8ce25e606c1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49" Jul 29 15:04:21.266379 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.266133 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsnqk\" (UniqueName: \"kubernetes.io/projected/b8545c81-83aa-4467-a427-d8ce25e606c1-kube-api-access-wsnqk\") pod \"payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49\" (UID: \"b8545c81-83aa-4467-a427-d8ce25e606c1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49" Jul 29 15:04:21.266690 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.266670 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8545c81-83aa-4467-a427-d8ce25e606c1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49\" (UID: \"b8545c81-83aa-4467-a427-d8ce25e606c1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49" Jul 29 15:04:21.266808 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.266764 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d62807e-8778-49f3-b1fb-a9d9286349da-plugins-config-volume\") pod \"payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb\" (UID: \"7d62807e-8778-49f3-b1fb-a9d9286349da\") " pod="openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb" Jul 29 15:04:21.273765 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.273741 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsnqk\" (UniqueName: \"kubernetes.io/projected/b8545c81-83aa-4467-a427-d8ce25e606c1-kube-api-access-wsnqk\") pod \"payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49\" (UID: \"b8545c81-83aa-4467-a427-d8ce25e606c1\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49" Jul 29 15:04:21.273852 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.273744 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7chzr\" (UniqueName: \"kubernetes.io/projected/7d62807e-8778-49f3-b1fb-a9d9286349da-kube-api-access-7chzr\") pod \"payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb\" (UID: \"7d62807e-8778-49f3-b1fb-a9d9286349da\") " pod="openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb" Jul 29 15:04:21.369442 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.369404 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49" Jul 29 15:04:21.375673 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.375653 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb" Jul 29 15:04:21.723410 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.723383 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49"] Jul 29 15:04:21.725483 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:04:21.725450 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8545c81_83aa_4467_a427_d8ce25e606c1.slice/crio-f7088ae3249b1ad527806638ae737e858b8d3d5fa8c196af6af765c852bec187 WatchSource:0}: Error finding container f7088ae3249b1ad527806638ae737e858b8d3d5fa8c196af6af765c852bec187: Status 404 returned error can't find the container with id f7088ae3249b1ad527806638ae737e858b8d3d5fa8c196af6af765c852bec187 Jul 29 15:04:21.752285 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:21.752255 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb"] Jul 29 15:04:21.758799 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:04:21.758776 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d62807e_8778_49f3_b1fb_a9d9286349da.slice/crio-95a5dc4545f939ae7900b11c28a11b85d7c3181e8edf1038f5fa1a6317ef8914 WatchSource:0}: Error finding container 95a5dc4545f939ae7900b11c28a11b85d7c3181e8edf1038f5fa1a6317ef8914: Status 404 returned error can't find the container with id 95a5dc4545f939ae7900b11c28a11b85d7c3181e8edf1038f5fa1a6317ef8914 Jul 29 15:04:22.673268 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:22.673144 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb" event={"ID":"7d62807e-8778-49f3-b1fb-a9d9286349da","Type":"ContainerStarted","Data":"aa0ebd549d9fb16f35a819dc4fce53beb22f9d0e3dddb89cab6f5e11f69d0b5d"} Jul 29 15:04:22.673268 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:22.673208 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb" event={"ID":"7d62807e-8778-49f3-b1fb-a9d9286349da","Type":"ContainerStarted","Data":"95a5dc4545f939ae7900b11c28a11b85d7c3181e8edf1038f5fa1a6317ef8914"} Jul 29 15:04:22.673268 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:22.673265 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb" Jul 29 15:04:22.675223 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:22.675201 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49" event={"ID":"b8545c81-83aa-4467-a427-d8ce25e606c1","Type":"ContainerStarted","Data":"7685a52a8d831f6013b674e5ca3bd28b8146e44a78fe97f81c8d64ed358bf09f"} Jul 29 15:04:22.675318 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:22.675241 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49" event={"ID":"b8545c81-83aa-4467-a427-d8ce25e606c1","Type":"ContainerStarted","Data":"f7088ae3249b1ad527806638ae737e858b8d3d5fa8c196af6af765c852bec187"} Jul 29 15:04:22.675318 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:22.675284 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49" Jul 29 15:04:22.689094 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:22.689054 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb" podStartSLOduration=1.689039847 podStartE2EDuration="1.689039847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:04:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:04:22.687884943 +0000 UTC m=+1739.416034133" watchObservedRunningTime="2026-07-29 15:04:22.689039847 +0000 UTC m=+1739.417189077" Jul 29 15:04:22.704175 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:22.704142 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49" podStartSLOduration=1.704130154 podStartE2EDuration="1.704130154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:04:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:04:22.702131679 +0000 UTC m=+1739.430280869" watchObservedRunningTime="2026-07-29 15:04:22.704130154 +0000 UTC m=+1739.432279343" Jul 29 15:04:33.687595 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:33.687562 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49" Jul 29 15:04:33.688014 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:33.687790 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb" Jul 29 15:04:51.137971 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.137916 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:51.198058 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.198017 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49"] Jul 29 15:04:51.198266 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.198229 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49" podUID="b8545c81-83aa-4467-a427-d8ce25e606c1" containerName="payload-pre-processing" containerID="cri-o://7685a52a8d831f6013b674e5ca3bd28b8146e44a78fe97f81c8d64ed358bf09f" gracePeriod=30 Jul 29 15:04:51.207853 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.207493 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb"] Jul 29 15:04:51.207853 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.207753 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb" podUID="7d62807e-8778-49f3-b1fb-a9d9286349da" containerName="payload-processing" containerID="cri-o://aa0ebd549d9fb16f35a819dc4fce53beb22f9d0e3dddb89cab6f5e11f69d0b5d" gracePeriod=30 Jul 29 15:04:51.288021 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.287983 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:51.480326 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.480298 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49" Jul 29 15:04:51.489188 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.489150 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:51.503837 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.503782 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb" Jul 29 15:04:51.523013 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.522975 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d62807e-8778-49f3-b1fb-a9d9286349da-plugins-config-volume\") pod \"7d62807e-8778-49f3-b1fb-a9d9286349da\" (UID: \"7d62807e-8778-49f3-b1fb-a9d9286349da\") " Jul 29 15:04:51.523208 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.523028 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7chzr\" (UniqueName: \"kubernetes.io/projected/7d62807e-8778-49f3-b1fb-a9d9286349da-kube-api-access-7chzr\") pod \"7d62807e-8778-49f3-b1fb-a9d9286349da\" (UID: \"7d62807e-8778-49f3-b1fb-a9d9286349da\") " Jul 29 15:04:51.523208 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.523113 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8545c81-83aa-4467-a427-d8ce25e606c1-plugins-config-volume\") pod \"b8545c81-83aa-4467-a427-d8ce25e606c1\" (UID: \"b8545c81-83aa-4467-a427-d8ce25e606c1\") " Jul 29 15:04:51.523208 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.523129 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsnqk\" (UniqueName: \"kubernetes.io/projected/b8545c81-83aa-4467-a427-d8ce25e606c1-kube-api-access-wsnqk\") pod \"b8545c81-83aa-4467-a427-d8ce25e606c1\" (UID: \"b8545c81-83aa-4467-a427-d8ce25e606c1\") " Jul 29 15:04:51.523484 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.523454 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d62807e-8778-49f3-b1fb-a9d9286349da-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7d62807e-8778-49f3-b1fb-a9d9286349da" (UID: "7d62807e-8778-49f3-b1fb-a9d9286349da"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:04:51.523589 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.523510 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8545c81-83aa-4467-a427-d8ce25e606c1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b8545c81-83aa-4467-a427-d8ce25e606c1" (UID: "b8545c81-83aa-4467-a427-d8ce25e606c1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:04:51.525301 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.525275 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8545c81-83aa-4467-a427-d8ce25e606c1-kube-api-access-wsnqk" (OuterVolumeSpecName: "kube-api-access-wsnqk") pod "b8545c81-83aa-4467-a427-d8ce25e606c1" (UID: "b8545c81-83aa-4467-a427-d8ce25e606c1"). InnerVolumeSpecName "kube-api-access-wsnqk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:04:51.525301 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.525285 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d62807e-8778-49f3-b1fb-a9d9286349da-kube-api-access-7chzr" (OuterVolumeSpecName: "kube-api-access-7chzr") pod "7d62807e-8778-49f3-b1fb-a9d9286349da" (UID: "7d62807e-8778-49f3-b1fb-a9d9286349da"). InnerVolumeSpecName "kube-api-access-7chzr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:04:51.623965 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.623934 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8545c81-83aa-4467-a427-d8ce25e606c1-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:04:51.623965 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.623961 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsnqk\" (UniqueName: \"kubernetes.io/projected/b8545c81-83aa-4467-a427-d8ce25e606c1-kube-api-access-wsnqk\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:04:51.623965 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.623971 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d62807e-8778-49f3-b1fb-a9d9286349da-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:04:51.624194 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.623980 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7chzr\" (UniqueName: \"kubernetes.io/projected/7d62807e-8778-49f3-b1fb-a9d9286349da-kube-api-access-7chzr\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:04:51.837161 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.837133 2580 generic.go:356] "Generic (PLEG): container finished" podID="7d62807e-8778-49f3-b1fb-a9d9286349da" containerID="aa0ebd549d9fb16f35a819dc4fce53beb22f9d0e3dddb89cab6f5e11f69d0b5d" exitCode=0 Jul 29 15:04:51.837272 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.837174 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb" Jul 29 15:04:51.837272 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.837219 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb" event={"ID":"7d62807e-8778-49f3-b1fb-a9d9286349da","Type":"ContainerDied","Data":"aa0ebd549d9fb16f35a819dc4fce53beb22f9d0e3dddb89cab6f5e11f69d0b5d"} Jul 29 15:04:51.837272 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.837265 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb" event={"ID":"7d62807e-8778-49f3-b1fb-a9d9286349da","Type":"ContainerDied","Data":"95a5dc4545f939ae7900b11c28a11b85d7c3181e8edf1038f5fa1a6317ef8914"} Jul 29 15:04:51.837446 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.837293 2580 scope.go:117] "RemoveContainer" containerID="aa0ebd549d9fb16f35a819dc4fce53beb22f9d0e3dddb89cab6f5e11f69d0b5d" Jul 29 15:04:51.838494 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.838475 2580 generic.go:356] "Generic (PLEG): container finished" podID="b8545c81-83aa-4467-a427-d8ce25e606c1" containerID="7685a52a8d831f6013b674e5ca3bd28b8146e44a78fe97f81c8d64ed358bf09f" exitCode=0 Jul 29 15:04:51.838589 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.838511 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49" Jul 29 15:04:51.838589 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.838525 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49" event={"ID":"b8545c81-83aa-4467-a427-d8ce25e606c1","Type":"ContainerDied","Data":"7685a52a8d831f6013b674e5ca3bd28b8146e44a78fe97f81c8d64ed358bf09f"} Jul 29 15:04:51.838589 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.838554 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49" event={"ID":"b8545c81-83aa-4467-a427-d8ce25e606c1","Type":"ContainerDied","Data":"f7088ae3249b1ad527806638ae737e858b8d3d5fa8c196af6af765c852bec187"} Jul 29 15:04:51.846404 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.846384 2580 scope.go:117] "RemoveContainer" containerID="aa0ebd549d9fb16f35a819dc4fce53beb22f9d0e3dddb89cab6f5e11f69d0b5d" Jul 29 15:04:51.846675 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:04:51.846656 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa0ebd549d9fb16f35a819dc4fce53beb22f9d0e3dddb89cab6f5e11f69d0b5d\": container with ID starting with aa0ebd549d9fb16f35a819dc4fce53beb22f9d0e3dddb89cab6f5e11f69d0b5d not found: ID does not exist" containerID="aa0ebd549d9fb16f35a819dc4fce53beb22f9d0e3dddb89cab6f5e11f69d0b5d" Jul 29 15:04:51.846763 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.846686 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa0ebd549d9fb16f35a819dc4fce53beb22f9d0e3dddb89cab6f5e11f69d0b5d"} err="failed to get container status \"aa0ebd549d9fb16f35a819dc4fce53beb22f9d0e3dddb89cab6f5e11f69d0b5d\": rpc error: code = NotFound desc = could not find container \"aa0ebd549d9fb16f35a819dc4fce53beb22f9d0e3dddb89cab6f5e11f69d0b5d\": container with ID starting with aa0ebd549d9fb16f35a819dc4fce53beb22f9d0e3dddb89cab6f5e11f69d0b5d not found: ID does not exist" Jul 29 15:04:51.846763 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.846710 2580 scope.go:117] "RemoveContainer" containerID="7685a52a8d831f6013b674e5ca3bd28b8146e44a78fe97f81c8d64ed358bf09f" Jul 29 15:04:51.855400 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.855374 2580 scope.go:117] "RemoveContainer" containerID="7685a52a8d831f6013b674e5ca3bd28b8146e44a78fe97f81c8d64ed358bf09f" Jul 29 15:04:51.855814 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:04:51.855784 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7685a52a8d831f6013b674e5ca3bd28b8146e44a78fe97f81c8d64ed358bf09f\": container with ID starting with 7685a52a8d831f6013b674e5ca3bd28b8146e44a78fe97f81c8d64ed358bf09f not found: ID does not exist" containerID="7685a52a8d831f6013b674e5ca3bd28b8146e44a78fe97f81c8d64ed358bf09f" Jul 29 15:04:51.855905 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.855825 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7685a52a8d831f6013b674e5ca3bd28b8146e44a78fe97f81c8d64ed358bf09f"} err="failed to get container status \"7685a52a8d831f6013b674e5ca3bd28b8146e44a78fe97f81c8d64ed358bf09f\": rpc error: code = NotFound desc = could not find container \"7685a52a8d831f6013b674e5ca3bd28b8146e44a78fe97f81c8d64ed358bf09f\": container with ID starting with 7685a52a8d831f6013b674e5ca3bd28b8146e44a78fe97f81c8d64ed358bf09f not found: ID does not exist" Jul 29 15:04:51.862048 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.862015 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49"] Jul 29 15:04:51.864945 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.864917 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-43963a57-5fcdcdcd76-dbm49"] Jul 29 15:04:51.873775 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.873752 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb"] Jul 29 15:04:51.884295 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:51.884277 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-43963a57-7d7d6cb68d-mljfb"] Jul 29 15:04:52.730221 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:52.730187 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:52.959410 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:52.959376 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:53.232447 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:53.232411 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:53.575301 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:53.575263 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:53.687131 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:53.687094 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:04:53.817420 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:53.817391 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d62807e-8778-49f3-b1fb-a9d9286349da" path="/var/lib/kubelet/pods/7d62807e-8778-49f3-b1fb-a9d9286349da/volumes" Jul 29 15:04:53.817806 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:04:53.817767 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8545c81-83aa-4467-a427-d8ce25e606c1" path="/var/lib/kubelet/pods/b8545c81-83aa-4467-a427-d8ce25e606c1/volumes" Jul 29 15:05:01.090161 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:01.090119 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:01.200166 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:01.200134 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:01.294423 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:01.294386 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:01.514454 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:01.514418 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:01.897583 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:01.897482 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:01.998230 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:01.998197 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:02.002033 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.001996 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg"] Jul 29 15:05:02.002500 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.002481 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8545c81-83aa-4467-a427-d8ce25e606c1" containerName="payload-pre-processing" Jul 29 15:05:02.002500 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.002501 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8545c81-83aa-4467-a427-d8ce25e606c1" containerName="payload-pre-processing" Jul 29 15:05:02.002664 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.002519 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d62807e-8778-49f3-b1fb-a9d9286349da" containerName="payload-processing" Jul 29 15:05:02.002664 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.002527 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d62807e-8778-49f3-b1fb-a9d9286349da" containerName="payload-processing" Jul 29 15:05:02.002664 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.002631 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d62807e-8778-49f3-b1fb-a9d9286349da" containerName="payload-processing" Jul 29 15:05:02.002664 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.002651 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8545c81-83aa-4467-a427-d8ce25e606c1" containerName="payload-pre-processing" Jul 29 15:05:02.005700 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.005672 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg" Jul 29 15:05:02.007689 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.007667 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-19551b81\"" Jul 29 15:05:02.015194 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.015167 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg"] Jul 29 15:05:02.033413 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.033380 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp"] Jul 29 15:05:02.036767 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.036714 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp" Jul 29 15:05:02.038818 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.038796 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-19551b81-dockercfg-9dslr\"" Jul 29 15:05:02.045442 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.045414 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp"] Jul 29 15:05:02.132472 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.132437 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw48d\" (UniqueName: \"kubernetes.io/projected/3e7b62db-9cea-48cf-a179-b7a7829f2782-kube-api-access-qw48d\") pod \"payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp\" (UID: \"3e7b62db-9cea-48cf-a179-b7a7829f2782\") " pod="openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp" Jul 29 15:05:02.132963 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.132540 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67df049b-bec1-4a0c-b717-4087ac426b50-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg\" (UID: \"67df049b-bec1-4a0c-b717-4087ac426b50\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg" Jul 29 15:05:02.132963 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.132604 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e7b62db-9cea-48cf-a179-b7a7829f2782-plugins-config-volume\") pod \"payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp\" (UID: \"3e7b62db-9cea-48cf-a179-b7a7829f2782\") " pod="openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp" Jul 29 15:05:02.132963 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.132763 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l48dr\" (UniqueName: \"kubernetes.io/projected/67df049b-bec1-4a0c-b717-4087ac426b50-kube-api-access-l48dr\") pod \"payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg\" (UID: \"67df049b-bec1-4a0c-b717-4087ac426b50\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg" Jul 29 15:05:02.234444 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.234336 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e7b62db-9cea-48cf-a179-b7a7829f2782-plugins-config-volume\") pod \"payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp\" (UID: \"3e7b62db-9cea-48cf-a179-b7a7829f2782\") " pod="openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp" Jul 29 15:05:02.234444 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.234407 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l48dr\" (UniqueName: \"kubernetes.io/projected/67df049b-bec1-4a0c-b717-4087ac426b50-kube-api-access-l48dr\") pod \"payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg\" (UID: \"67df049b-bec1-4a0c-b717-4087ac426b50\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg" Jul 29 15:05:02.234444 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.234440 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw48d\" (UniqueName: \"kubernetes.io/projected/3e7b62db-9cea-48cf-a179-b7a7829f2782-kube-api-access-qw48d\") pod \"payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp\" (UID: \"3e7b62db-9cea-48cf-a179-b7a7829f2782\") " pod="openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp" Jul 29 15:05:02.234792 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.234466 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67df049b-bec1-4a0c-b717-4087ac426b50-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg\" (UID: \"67df049b-bec1-4a0c-b717-4087ac426b50\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg" Jul 29 15:05:02.235234 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.235206 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e7b62db-9cea-48cf-a179-b7a7829f2782-plugins-config-volume\") pod \"payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp\" (UID: \"3e7b62db-9cea-48cf-a179-b7a7829f2782\") " pod="openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp" Jul 29 15:05:02.235325 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.235234 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67df049b-bec1-4a0c-b717-4087ac426b50-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg\" (UID: \"67df049b-bec1-4a0c-b717-4087ac426b50\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg" Jul 29 15:05:02.248985 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.248949 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l48dr\" (UniqueName: \"kubernetes.io/projected/67df049b-bec1-4a0c-b717-4087ac426b50-kube-api-access-l48dr\") pod \"payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg\" (UID: \"67df049b-bec1-4a0c-b717-4087ac426b50\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg" Jul 29 15:05:02.249121 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.249020 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw48d\" (UniqueName: \"kubernetes.io/projected/3e7b62db-9cea-48cf-a179-b7a7829f2782-kube-api-access-qw48d\") pod \"payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp\" (UID: \"3e7b62db-9cea-48cf-a179-b7a7829f2782\") " pod="openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp" Jul 29 15:05:02.317867 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.317826 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg" Jul 29 15:05:02.349684 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.349639 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp" Jul 29 15:05:02.485905 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.485814 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg"] Jul 29 15:05:02.487128 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:05:02.487091 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67df049b_bec1_4a0c_b717_4087ac426b50.slice/crio-4defc91e4dc317bfb948f473eb1dc77c8ae3621cd0095c87043f8645bc750808 WatchSource:0}: Error finding container 4defc91e4dc317bfb948f473eb1dc77c8ae3621cd0095c87043f8645bc750808: Status 404 returned error can't find the container with id 4defc91e4dc317bfb948f473eb1dc77c8ae3621cd0095c87043f8645bc750808 Jul 29 15:05:02.515678 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.515653 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp"] Jul 29 15:05:02.519083 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:05:02.519053 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e7b62db_9cea_48cf_a179_b7a7829f2782.slice/crio-2fe333425d507d1586ca3e01dd5257655cb17033ba0a0ca16383101345405a1e WatchSource:0}: Error finding container 2fe333425d507d1586ca3e01dd5257655cb17033ba0a0ca16383101345405a1e: Status 404 returned error can't find the container with id 2fe333425d507d1586ca3e01dd5257655cb17033ba0a0ca16383101345405a1e Jul 29 15:05:02.901793 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.901755 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg" event={"ID":"67df049b-bec1-4a0c-b717-4087ac426b50","Type":"ContainerStarted","Data":"75577fd144148d06f71919504f4ff3b05f248074bca72708c5b89245b85f054c"} Jul 29 15:05:02.902026 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.901826 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg" Jul 29 15:05:02.902026 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.901850 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg" event={"ID":"67df049b-bec1-4a0c-b717-4087ac426b50","Type":"ContainerStarted","Data":"4defc91e4dc317bfb948f473eb1dc77c8ae3621cd0095c87043f8645bc750808"} Jul 29 15:05:02.903380 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.903353 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp" event={"ID":"3e7b62db-9cea-48cf-a179-b7a7829f2782","Type":"ContainerStarted","Data":"c5e2c57854005a76fcef367549f8089c9bbe9acdcf95403c1f4d85e036b7f6a9"} Jul 29 15:05:02.903520 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.903402 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp" Jul 29 15:05:02.903520 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.903419 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp" event={"ID":"3e7b62db-9cea-48cf-a179-b7a7829f2782","Type":"ContainerStarted","Data":"2fe333425d507d1586ca3e01dd5257655cb17033ba0a0ca16383101345405a1e"} Jul 29 15:05:02.918342 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.918283 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg" podStartSLOduration=1.918263364 podStartE2EDuration="1.918263364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:05:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:05:02.916172217 +0000 UTC m=+1779.644321397" watchObservedRunningTime="2026-07-29 15:05:02.918263364 +0000 UTC m=+1779.646412555" Jul 29 15:05:02.935425 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:02.935359 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp" podStartSLOduration=0.935340266 podStartE2EDuration="935.340266ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:05:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:05:02.934630435 +0000 UTC m=+1779.662779629" watchObservedRunningTime="2026-07-29 15:05:02.935340266 +0000 UTC m=+1779.663489456" Jul 29 15:05:13.911842 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:13.911810 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp" Jul 29 15:05:13.912240 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:13.911993 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg" Jul 29 15:05:23.853130 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:23.853102 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 15:05:23.856231 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:23.856210 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 15:05:31.819479 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:31.819444 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:31.851297 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:31.851262 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp"] Jul 29 15:05:31.851512 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:31.851487 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp" podUID="3e7b62db-9cea-48cf-a179-b7a7829f2782" containerName="payload-processing" containerID="cri-o://c5e2c57854005a76fcef367549f8089c9bbe9acdcf95403c1f4d85e036b7f6a9" gracePeriod=30 Jul 29 15:05:31.857638 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:31.857575 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg"] Jul 29 15:05:31.857976 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:31.857950 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg" podUID="67df049b-bec1-4a0c-b717-4087ac426b50" containerName="payload-pre-processing" containerID="cri-o://75577fd144148d06f71919504f4ff3b05f248074bca72708c5b89245b85f054c" gracePeriod=30 Jul 29 15:05:31.887949 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:31.887919 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:31.952624 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:31.952586 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:32.012753 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:32.012441 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:32.072179 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:32.072098 2580 generic.go:356] "Generic (PLEG): container finished" podID="3e7b62db-9cea-48cf-a179-b7a7829f2782" containerID="c5e2c57854005a76fcef367549f8089c9bbe9acdcf95403c1f4d85e036b7f6a9" exitCode=0 Jul 29 15:05:32.072179 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:32.072159 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp" event={"ID":"3e7b62db-9cea-48cf-a179-b7a7829f2782","Type":"ContainerDied","Data":"c5e2c57854005a76fcef367549f8089c9bbe9acdcf95403c1f4d85e036b7f6a9"} Jul 29 15:05:32.074127 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:32.074104 2580 generic.go:356] "Generic (PLEG): container finished" podID="67df049b-bec1-4a0c-b717-4087ac426b50" containerID="75577fd144148d06f71919504f4ff3b05f248074bca72708c5b89245b85f054c" exitCode=0 Jul 29 15:05:32.074249 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:32.074166 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg" event={"ID":"67df049b-bec1-4a0c-b717-4087ac426b50","Type":"ContainerDied","Data":"75577fd144148d06f71919504f4ff3b05f248074bca72708c5b89245b85f054c"} Jul 29 15:05:32.112905 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:32.112884 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp" Jul 29 15:05:32.142665 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:32.142637 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg" Jul 29 15:05:32.193553 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:32.193529 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l48dr\" (UniqueName: \"kubernetes.io/projected/67df049b-bec1-4a0c-b717-4087ac426b50-kube-api-access-l48dr\") pod \"67df049b-bec1-4a0c-b717-4087ac426b50\" (UID: \"67df049b-bec1-4a0c-b717-4087ac426b50\") " Jul 29 15:05:32.193682 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:32.193566 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67df049b-bec1-4a0c-b717-4087ac426b50-plugins-config-volume\") pod \"67df049b-bec1-4a0c-b717-4087ac426b50\" (UID: \"67df049b-bec1-4a0c-b717-4087ac426b50\") " Jul 29 15:05:32.193682 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:32.193585 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e7b62db-9cea-48cf-a179-b7a7829f2782-plugins-config-volume\") pod \"3e7b62db-9cea-48cf-a179-b7a7829f2782\" (UID: \"3e7b62db-9cea-48cf-a179-b7a7829f2782\") " Jul 29 15:05:32.193682 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:32.193629 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qw48d\" (UniqueName: \"kubernetes.io/projected/3e7b62db-9cea-48cf-a179-b7a7829f2782-kube-api-access-qw48d\") pod \"3e7b62db-9cea-48cf-a179-b7a7829f2782\" (UID: \"3e7b62db-9cea-48cf-a179-b7a7829f2782\") " Jul 29 15:05:32.194069 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:32.194039 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e7b62db-9cea-48cf-a179-b7a7829f2782-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3e7b62db-9cea-48cf-a179-b7a7829f2782" (UID: "3e7b62db-9cea-48cf-a179-b7a7829f2782"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:05:32.194069 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:32.194056 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67df049b-bec1-4a0c-b717-4087ac426b50-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "67df049b-bec1-4a0c-b717-4087ac426b50" (UID: "67df049b-bec1-4a0c-b717-4087ac426b50"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:05:32.195589 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:32.195571 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67df049b-bec1-4a0c-b717-4087ac426b50-kube-api-access-l48dr" (OuterVolumeSpecName: "kube-api-access-l48dr") pod "67df049b-bec1-4a0c-b717-4087ac426b50" (UID: "67df049b-bec1-4a0c-b717-4087ac426b50"). InnerVolumeSpecName "kube-api-access-l48dr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:05:32.195683 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:32.195643 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e7b62db-9cea-48cf-a179-b7a7829f2782-kube-api-access-qw48d" (OuterVolumeSpecName: "kube-api-access-qw48d") pod "3e7b62db-9cea-48cf-a179-b7a7829f2782" (UID: "3e7b62db-9cea-48cf-a179-b7a7829f2782"). InnerVolumeSpecName "kube-api-access-qw48d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:05:32.294678 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:32.294648 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l48dr\" (UniqueName: \"kubernetes.io/projected/67df049b-bec1-4a0c-b717-4087ac426b50-kube-api-access-l48dr\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:05:32.294678 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:32.294673 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/67df049b-bec1-4a0c-b717-4087ac426b50-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:05:32.294678 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:32.294682 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e7b62db-9cea-48cf-a179-b7a7829f2782-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:05:32.294915 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:32.294692 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qw48d\" (UniqueName: \"kubernetes.io/projected/3e7b62db-9cea-48cf-a179-b7a7829f2782-kube-api-access-qw48d\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:05:32.373395 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:32.373297 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:32.838744 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:32.838686 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:33.080872 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:33.080839 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg" Jul 29 15:05:33.081017 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:33.080871 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg" event={"ID":"67df049b-bec1-4a0c-b717-4087ac426b50","Type":"ContainerDied","Data":"4defc91e4dc317bfb948f473eb1dc77c8ae3621cd0095c87043f8645bc750808"} Jul 29 15:05:33.081017 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:33.080925 2580 scope.go:117] "RemoveContainer" containerID="75577fd144148d06f71919504f4ff3b05f248074bca72708c5b89245b85f054c" Jul 29 15:05:33.082281 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:33.082230 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp" event={"ID":"3e7b62db-9cea-48cf-a179-b7a7829f2782","Type":"ContainerDied","Data":"2fe333425d507d1586ca3e01dd5257655cb17033ba0a0ca16383101345405a1e"} Jul 29 15:05:33.082281 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:33.082268 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp" Jul 29 15:05:33.091443 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:33.091428 2580 scope.go:117] "RemoveContainer" containerID="c5e2c57854005a76fcef367549f8089c9bbe9acdcf95403c1f4d85e036b7f6a9" Jul 29 15:05:33.102902 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:33.102884 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg"] Jul 29 15:05:33.106471 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:33.106447 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-19551b81-f88655f6-5sxfg"] Jul 29 15:05:33.116450 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:33.116427 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp"] Jul 29 15:05:33.122537 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:33.122520 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-19551b81-6f7c974586-l9kdp"] Jul 29 15:05:33.758879 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:33.758835 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:33.817824 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:33.817794 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e7b62db-9cea-48cf-a179-b7a7829f2782" path="/var/lib/kubelet/pods/3e7b62db-9cea-48cf-a179-b7a7829f2782/volumes" Jul 29 15:05:33.818133 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:33.818120 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67df049b-bec1-4a0c-b717-4087ac426b50" path="/var/lib/kubelet/pods/67df049b-bec1-4a0c-b717-4087ac426b50/volumes" Jul 29 15:05:33.855115 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:33.855089 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:33.957393 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:33.957363 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:35.434761 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:35.434708 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:41.608416 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:41.608374 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:41.716039 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:41.716009 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:41.808645 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:41.808610 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:41.910223 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:41.910154 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:05:42.347525 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:05:42.347482 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:08.348156 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:06:08.348112 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:15.312422 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:06:15.312378 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:15.410532 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:06:15.410492 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:30.767830 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:06:30.767791 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:36.610329 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:06:36.610290 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:36.722212 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:06:36.722175 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:36.844283 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:06:36.844251 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:47.356173 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:06:47.356082 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:48.405596 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:06:48.405550 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:54.570569 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:06:54.570533 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:54.671923 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:06:54.671889 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:54.771790 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:06:54.771762 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:06:54.870885 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:06:54.870812 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:06.339867 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:06.339820 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:06.438002 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:06.437962 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:06.537641 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:06.537611 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:08.294938 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:08.294880 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:14.450996 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:14.450968 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:14.566663 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:14.566638 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:15.261455 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:15.261409 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:21.233740 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:21.233687 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:21.334240 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:21.334204 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:21.469513 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:21.469479 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:21.769235 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:21.769191 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:21.994506 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:21.994466 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:24.603635 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:24.603600 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:24.722024 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:24.721983 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:24.815166 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:24.815129 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:25.321111 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.321074 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:25.420775 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.420707 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:25.437706 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.437677 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w"] Jul 29 15:07:25.438053 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.438041 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e7b62db-9cea-48cf-a179-b7a7829f2782" containerName="payload-processing" Jul 29 15:07:25.438097 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.438055 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e7b62db-9cea-48cf-a179-b7a7829f2782" containerName="payload-processing" Jul 29 15:07:25.438097 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.438078 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67df049b-bec1-4a0c-b717-4087ac426b50" containerName="payload-pre-processing" Jul 29 15:07:25.438097 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.438083 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="67df049b-bec1-4a0c-b717-4087ac426b50" containerName="payload-pre-processing" Jul 29 15:07:25.438195 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.438139 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e7b62db-9cea-48cf-a179-b7a7829f2782" containerName="payload-processing" Jul 29 15:07:25.438195 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.438147 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="67df049b-bec1-4a0c-b717-4087ac426b50" containerName="payload-pre-processing" Jul 29 15:07:25.446265 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.446026 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w" Jul 29 15:07:25.449567 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.449540 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w"] Jul 29 15:07:25.450274 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.450246 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-0af43e8c\"" Jul 29 15:07:25.469774 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.469749 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2"] Jul 29 15:07:25.473425 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.473408 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2" Jul 29 15:07:25.475456 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.475437 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-0af43e8c-dockercfg-jbf4t\"" Jul 29 15:07:25.480579 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.480556 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5656k\" (UniqueName: \"kubernetes.io/projected/b87a9486-d55d-4276-9d5a-31a3fcc6f1aa-kube-api-access-5656k\") pod \"payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w\" (UID: \"b87a9486-d55d-4276-9d5a-31a3fcc6f1aa\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w" Jul 29 15:07:25.480708 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.480683 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b87a9486-d55d-4276-9d5a-31a3fcc6f1aa-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w\" (UID: \"b87a9486-d55d-4276-9d5a-31a3fcc6f1aa\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w" Jul 29 15:07:25.484709 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.484690 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2"] Jul 29 15:07:25.581956 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.581882 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5656k\" (UniqueName: \"kubernetes.io/projected/b87a9486-d55d-4276-9d5a-31a3fcc6f1aa-kube-api-access-5656k\") pod \"payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w\" (UID: \"b87a9486-d55d-4276-9d5a-31a3fcc6f1aa\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w" Jul 29 15:07:25.581956 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.581939 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b87a9486-d55d-4276-9d5a-31a3fcc6f1aa-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w\" (UID: \"b87a9486-d55d-4276-9d5a-31a3fcc6f1aa\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w" Jul 29 15:07:25.582151 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.581988 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a4d7ddd-9211-42f9-b523-fd16ccafe146-plugins-config-volume\") pod \"payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2\" (UID: \"9a4d7ddd-9211-42f9-b523-fd16ccafe146\") " pod="openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2" Jul 29 15:07:25.582151 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.582013 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mrt9\" (UniqueName: \"kubernetes.io/projected/9a4d7ddd-9211-42f9-b523-fd16ccafe146-kube-api-access-8mrt9\") pod \"payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2\" (UID: \"9a4d7ddd-9211-42f9-b523-fd16ccafe146\") " pod="openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2" Jul 29 15:07:25.582574 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.582554 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b87a9486-d55d-4276-9d5a-31a3fcc6f1aa-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w\" (UID: \"b87a9486-d55d-4276-9d5a-31a3fcc6f1aa\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w" Jul 29 15:07:25.589414 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.589391 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5656k\" (UniqueName: \"kubernetes.io/projected/b87a9486-d55d-4276-9d5a-31a3fcc6f1aa-kube-api-access-5656k\") pod \"payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w\" (UID: \"b87a9486-d55d-4276-9d5a-31a3fcc6f1aa\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w" Jul 29 15:07:25.682773 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.682717 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a4d7ddd-9211-42f9-b523-fd16ccafe146-plugins-config-volume\") pod \"payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2\" (UID: \"9a4d7ddd-9211-42f9-b523-fd16ccafe146\") " pod="openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2" Jul 29 15:07:25.682773 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.682778 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mrt9\" (UniqueName: \"kubernetes.io/projected/9a4d7ddd-9211-42f9-b523-fd16ccafe146-kube-api-access-8mrt9\") pod \"payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2\" (UID: \"9a4d7ddd-9211-42f9-b523-fd16ccafe146\") " pod="openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2" Jul 29 15:07:25.683374 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.683355 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a4d7ddd-9211-42f9-b523-fd16ccafe146-plugins-config-volume\") pod \"payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2\" (UID: \"9a4d7ddd-9211-42f9-b523-fd16ccafe146\") " pod="openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2" Jul 29 15:07:25.691396 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.691340 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mrt9\" (UniqueName: \"kubernetes.io/projected/9a4d7ddd-9211-42f9-b523-fd16ccafe146-kube-api-access-8mrt9\") pod \"payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2\" (UID: \"9a4d7ddd-9211-42f9-b523-fd16ccafe146\") " pod="openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2" Jul 29 15:07:25.761640 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.761611 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w" Jul 29 15:07:25.784256 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.784228 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2" Jul 29 15:07:25.926662 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.926637 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w"] Jul 29 15:07:25.928858 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:07:25.928829 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb87a9486_d55d_4276_9d5a_31a3fcc6f1aa.slice/crio-3a23fa33a5899caec15c310d25aedef21646d811dba2606be447fc46c8bac736 WatchSource:0}: Error finding container 3a23fa33a5899caec15c310d25aedef21646d811dba2606be447fc46c8bac736: Status 404 returned error can't find the container with id 3a23fa33a5899caec15c310d25aedef21646d811dba2606be447fc46c8bac736 Jul 29 15:07:25.966615 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:25.966589 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2"] Jul 29 15:07:25.967824 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:07:25.967788 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a4d7ddd_9211_42f9_b523_fd16ccafe146.slice/crio-e363fb83f30a18ab513dc68f122d5e01c3649d7871628d77ab470719baaa6688 WatchSource:0}: Error finding container e363fb83f30a18ab513dc68f122d5e01c3649d7871628d77ab470719baaa6688: Status 404 returned error can't find the container with id e363fb83f30a18ab513dc68f122d5e01c3649d7871628d77ab470719baaa6688 Jul 29 15:07:26.706359 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:26.706326 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2" event={"ID":"9a4d7ddd-9211-42f9-b523-fd16ccafe146","Type":"ContainerStarted","Data":"5f3f89dbd0c47d2013f3643384f4f22b8923069911e82774ef6ac8b7c2d38c57"} Jul 29 15:07:26.706838 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:26.706377 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2" event={"ID":"9a4d7ddd-9211-42f9-b523-fd16ccafe146","Type":"ContainerStarted","Data":"e363fb83f30a18ab513dc68f122d5e01c3649d7871628d77ab470719baaa6688"} Jul 29 15:07:26.706838 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:26.706432 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2" Jul 29 15:07:26.708031 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:26.708010 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w" event={"ID":"b87a9486-d55d-4276-9d5a-31a3fcc6f1aa","Type":"ContainerStarted","Data":"7210b982248bc895c950d63efb338b980cd3c9e01818e4bca3b69b5d52cdee14"} Jul 29 15:07:26.708122 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:26.708047 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w" event={"ID":"b87a9486-d55d-4276-9d5a-31a3fcc6f1aa","Type":"ContainerStarted","Data":"3a23fa33a5899caec15c310d25aedef21646d811dba2606be447fc46c8bac736"} Jul 29 15:07:26.708122 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:26.708094 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w" Jul 29 15:07:26.724263 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:26.724224 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2" podStartSLOduration=1.724211581 podStartE2EDuration="1.724211581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:07:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:07:26.721286385 +0000 UTC m=+1923.449435574" watchObservedRunningTime="2026-07-29 15:07:26.724211581 +0000 UTC m=+1923.452360771" Jul 29 15:07:26.738861 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:26.738813 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w" podStartSLOduration=1.738798825 podStartE2EDuration="1.738798825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:07:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:07:26.736362891 +0000 UTC m=+1923.464512082" watchObservedRunningTime="2026-07-29 15:07:26.738798825 +0000 UTC m=+1923.466948072" Jul 29 15:07:37.715448 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:37.715419 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2" Jul 29 15:07:37.716036 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:37.715539 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w" Jul 29 15:07:49.522102 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:49.522051 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:50.183120 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:50.183082 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:50.280376 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:50.280337 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:53.481934 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:53.481891 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:56.629194 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:56.629162 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:07:56.729461 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:07:56.729423 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:05.078114 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:05.078076 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:05.178874 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:05.178836 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:07.523792 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.523757 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:07.544811 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.544781 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2"] Jul 29 15:08:07.545036 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.544989 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2" podUID="9a4d7ddd-9211-42f9-b523-fd16ccafe146" containerName="payload-processing" containerID="cri-o://5f3f89dbd0c47d2013f3643384f4f22b8923069911e82774ef6ac8b7c2d38c57" gracePeriod=30 Jul 29 15:08:07.556910 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.553569 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w"] Jul 29 15:08:07.556910 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.553889 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w" podUID="b87a9486-d55d-4276-9d5a-31a3fcc6f1aa" containerName="payload-pre-processing" containerID="cri-o://7210b982248bc895c950d63efb338b980cd3c9e01818e4bca3b69b5d52cdee14" gracePeriod=30 Jul 29 15:08:07.624764 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.624709 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:07.714337 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.714025 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2" podUID="9a4d7ddd-9211-42f9-b523-fd16ccafe146" containerName="payload-processing" probeResult="failure" output="dial tcp 10.133.0.60:9004: connect: connection refused" Jul 29 15:08:07.714337 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.714114 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w" podUID="b87a9486-d55d-4276-9d5a-31a3fcc6f1aa" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.59:9004: connect: connection refused" Jul 29 15:08:07.716300 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.716270 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:07.808067 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.808043 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2" Jul 29 15:08:07.839953 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.839914 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:07.841609 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.841592 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w" Jul 29 15:08:07.936359 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.936323 2580 generic.go:356] "Generic (PLEG): container finished" podID="b87a9486-d55d-4276-9d5a-31a3fcc6f1aa" containerID="7210b982248bc895c950d63efb338b980cd3c9e01818e4bca3b69b5d52cdee14" exitCode=0 Jul 29 15:08:07.936559 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.936383 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w" Jul 29 15:08:07.936559 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.936411 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w" event={"ID":"b87a9486-d55d-4276-9d5a-31a3fcc6f1aa","Type":"ContainerDied","Data":"7210b982248bc895c950d63efb338b980cd3c9e01818e4bca3b69b5d52cdee14"} Jul 29 15:08:07.936559 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.936506 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w" event={"ID":"b87a9486-d55d-4276-9d5a-31a3fcc6f1aa","Type":"ContainerDied","Data":"3a23fa33a5899caec15c310d25aedef21646d811dba2606be447fc46c8bac736"} Jul 29 15:08:07.936698 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.936528 2580 scope.go:117] "RemoveContainer" containerID="7210b982248bc895c950d63efb338b980cd3c9e01818e4bca3b69b5d52cdee14" Jul 29 15:08:07.937713 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.937694 2580 generic.go:356] "Generic (PLEG): container finished" podID="9a4d7ddd-9211-42f9-b523-fd16ccafe146" containerID="5f3f89dbd0c47d2013f3643384f4f22b8923069911e82774ef6ac8b7c2d38c57" exitCode=0 Jul 29 15:08:07.937827 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.937794 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2" Jul 29 15:08:07.937873 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.937794 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2" event={"ID":"9a4d7ddd-9211-42f9-b523-fd16ccafe146","Type":"ContainerDied","Data":"5f3f89dbd0c47d2013f3643384f4f22b8923069911e82774ef6ac8b7c2d38c57"} Jul 29 15:08:07.937917 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.937904 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2" event={"ID":"9a4d7ddd-9211-42f9-b523-fd16ccafe146","Type":"ContainerDied","Data":"e363fb83f30a18ab513dc68f122d5e01c3649d7871628d77ab470719baaa6688"} Jul 29 15:08:07.945181 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.945165 2580 scope.go:117] "RemoveContainer" containerID="7210b982248bc895c950d63efb338b980cd3c9e01818e4bca3b69b5d52cdee14" Jul 29 15:08:07.945443 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:08:07.945427 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7210b982248bc895c950d63efb338b980cd3c9e01818e4bca3b69b5d52cdee14\": container with ID starting with 7210b982248bc895c950d63efb338b980cd3c9e01818e4bca3b69b5d52cdee14 not found: ID does not exist" containerID="7210b982248bc895c950d63efb338b980cd3c9e01818e4bca3b69b5d52cdee14" Jul 29 15:08:07.945492 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.945451 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7210b982248bc895c950d63efb338b980cd3c9e01818e4bca3b69b5d52cdee14"} err="failed to get container status \"7210b982248bc895c950d63efb338b980cd3c9e01818e4bca3b69b5d52cdee14\": rpc error: code = NotFound desc = could not find container \"7210b982248bc895c950d63efb338b980cd3c9e01818e4bca3b69b5d52cdee14\": container with ID starting with 7210b982248bc895c950d63efb338b980cd3c9e01818e4bca3b69b5d52cdee14 not found: ID does not exist" Jul 29 15:08:07.945492 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.945466 2580 scope.go:117] "RemoveContainer" containerID="5f3f89dbd0c47d2013f3643384f4f22b8923069911e82774ef6ac8b7c2d38c57" Jul 29 15:08:07.954937 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.954851 2580 scope.go:117] "RemoveContainer" containerID="5f3f89dbd0c47d2013f3643384f4f22b8923069911e82774ef6ac8b7c2d38c57" Jul 29 15:08:07.955181 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:08:07.955159 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f3f89dbd0c47d2013f3643384f4f22b8923069911e82774ef6ac8b7c2d38c57\": container with ID starting with 5f3f89dbd0c47d2013f3643384f4f22b8923069911e82774ef6ac8b7c2d38c57 not found: ID does not exist" containerID="5f3f89dbd0c47d2013f3643384f4f22b8923069911e82774ef6ac8b7c2d38c57" Jul 29 15:08:07.955238 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.955186 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f3f89dbd0c47d2013f3643384f4f22b8923069911e82774ef6ac8b7c2d38c57"} err="failed to get container status \"5f3f89dbd0c47d2013f3643384f4f22b8923069911e82774ef6ac8b7c2d38c57\": rpc error: code = NotFound desc = could not find container \"5f3f89dbd0c47d2013f3643384f4f22b8923069911e82774ef6ac8b7c2d38c57\": container with ID starting with 5f3f89dbd0c47d2013f3643384f4f22b8923069911e82774ef6ac8b7c2d38c57 not found: ID does not exist" Jul 29 15:08:07.956412 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.956396 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a4d7ddd-9211-42f9-b523-fd16ccafe146-plugins-config-volume\") pod \"9a4d7ddd-9211-42f9-b523-fd16ccafe146\" (UID: \"9a4d7ddd-9211-42f9-b523-fd16ccafe146\") " Jul 29 15:08:07.956462 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.956424 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b87a9486-d55d-4276-9d5a-31a3fcc6f1aa-plugins-config-volume\") pod \"b87a9486-d55d-4276-9d5a-31a3fcc6f1aa\" (UID: \"b87a9486-d55d-4276-9d5a-31a3fcc6f1aa\") " Jul 29 15:08:07.956462 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.956442 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5656k\" (UniqueName: \"kubernetes.io/projected/b87a9486-d55d-4276-9d5a-31a3fcc6f1aa-kube-api-access-5656k\") pod \"b87a9486-d55d-4276-9d5a-31a3fcc6f1aa\" (UID: \"b87a9486-d55d-4276-9d5a-31a3fcc6f1aa\") " Jul 29 15:08:07.956545 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.956524 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mrt9\" (UniqueName: \"kubernetes.io/projected/9a4d7ddd-9211-42f9-b523-fd16ccafe146-kube-api-access-8mrt9\") pod \"9a4d7ddd-9211-42f9-b523-fd16ccafe146\" (UID: \"9a4d7ddd-9211-42f9-b523-fd16ccafe146\") " Jul 29 15:08:07.956824 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.956798 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a4d7ddd-9211-42f9-b523-fd16ccafe146-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9a4d7ddd-9211-42f9-b523-fd16ccafe146" (UID: "9a4d7ddd-9211-42f9-b523-fd16ccafe146"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:08:07.956909 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.956824 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b87a9486-d55d-4276-9d5a-31a3fcc6f1aa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b87a9486-d55d-4276-9d5a-31a3fcc6f1aa" (UID: "b87a9486-d55d-4276-9d5a-31a3fcc6f1aa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:08:07.958421 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.958400 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b87a9486-d55d-4276-9d5a-31a3fcc6f1aa-kube-api-access-5656k" (OuterVolumeSpecName: "kube-api-access-5656k") pod "b87a9486-d55d-4276-9d5a-31a3fcc6f1aa" (UID: "b87a9486-d55d-4276-9d5a-31a3fcc6f1aa"). InnerVolumeSpecName "kube-api-access-5656k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:08:07.958515 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:07.958493 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a4d7ddd-9211-42f9-b523-fd16ccafe146-kube-api-access-8mrt9" (OuterVolumeSpecName: "kube-api-access-8mrt9") pod "9a4d7ddd-9211-42f9-b523-fd16ccafe146" (UID: "9a4d7ddd-9211-42f9-b523-fd16ccafe146"). InnerVolumeSpecName "kube-api-access-8mrt9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:08:08.057311 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:08.057218 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mrt9\" (UniqueName: \"kubernetes.io/projected/9a4d7ddd-9211-42f9-b523-fd16ccafe146-kube-api-access-8mrt9\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:08:08.057311 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:08.057247 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a4d7ddd-9211-42f9-b523-fd16ccafe146-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:08:08.057311 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:08.057257 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b87a9486-d55d-4276-9d5a-31a3fcc6f1aa-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:08:08.057311 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:08.057266 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5656k\" (UniqueName: \"kubernetes.io/projected/b87a9486-d55d-4276-9d5a-31a3fcc6f1aa-kube-api-access-5656k\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:08:08.105176 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:08.105139 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:08.262478 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:08.262446 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w"] Jul 29 15:08:08.265916 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:08.265893 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0af43e8c-85d879ff44-z498w"] Jul 29 15:08:08.276293 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:08.276265 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2"] Jul 29 15:08:08.280170 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:08.280145 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0af43e8c-6c7c66494-tn5h2"] Jul 29 15:08:09.818371 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:09.818338 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a4d7ddd-9211-42f9-b523-fd16ccafe146" path="/var/lib/kubelet/pods/9a4d7ddd-9211-42f9-b523-fd16ccafe146/volumes" Jul 29 15:08:09.818844 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:09.818684 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b87a9486-d55d-4276-9d5a-31a3fcc6f1aa" path="/var/lib/kubelet/pods/b87a9486-d55d-4276-9d5a-31a3fcc6f1aa/volumes" Jul 29 15:08:13.927452 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:13.927393 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:14.005117 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:14.005084 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:14.221376 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:14.221255 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:14.447522 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:14.447484 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:14.578245 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:14.578217 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:14.676261 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:14.676230 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:14.800646 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:14.800619 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:20.505125 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:20.505089 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:20.565953 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:20.565920 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:20.664094 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:20.664055 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:20.763061 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:20.762994 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:20.861831 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:20.861807 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:26.873880 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:26.873841 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:26.958608 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:26.958559 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:28.507112 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:28.507071 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:28.608118 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:28.608084 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:33.891145 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:33.891105 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:34.218192 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:34.218093 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:34.648048 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:34.648010 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:36.883921 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:36.883889 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:36.998081 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:36.997985 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:37.355055 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:37.355019 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:37.456168 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:37.456139 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:37.876295 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:37.876237 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:37.980976 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:37.980946 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:08:37.985515 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:37.985485 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt"] Jul 29 15:08:37.985865 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:37.985852 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a4d7ddd-9211-42f9-b523-fd16ccafe146" containerName="payload-processing" Jul 29 15:08:37.985915 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:37.985866 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a4d7ddd-9211-42f9-b523-fd16ccafe146" containerName="payload-processing" Jul 29 15:08:37.985915 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:37.985878 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b87a9486-d55d-4276-9d5a-31a3fcc6f1aa" containerName="payload-pre-processing" Jul 29 15:08:37.985915 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:37.985884 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b87a9486-d55d-4276-9d5a-31a3fcc6f1aa" containerName="payload-pre-processing" Jul 29 15:08:37.986013 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:37.985938 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b87a9486-d55d-4276-9d5a-31a3fcc6f1aa" containerName="payload-pre-processing" Jul 29 15:08:37.986013 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:37.985949 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a4d7ddd-9211-42f9-b523-fd16ccafe146" containerName="payload-processing" Jul 29 15:08:37.988950 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:37.988934 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt" Jul 29 15:08:37.991413 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:37.991397 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-a385d8ab-dockercfg-nrtpb\"" Jul 29 15:08:37.991745 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:37.991713 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-a385d8ab\"" Jul 29 15:08:38.006452 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:38.006432 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt"] Jul 29 15:08:38.029266 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:38.029247 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp"] Jul 29 15:08:38.032808 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:38.032784 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp" Jul 29 15:08:38.051615 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:38.051593 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp"] Jul 29 15:08:38.074437 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:38.074417 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2bhq\" (UniqueName: \"kubernetes.io/projected/f4eaec18-2534-486e-8d53-e28558cf9be0-kube-api-access-n2bhq\") pod \"payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp\" (UID: \"f4eaec18-2534-486e-8d53-e28558cf9be0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp" Jul 29 15:08:38.074554 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:38.074448 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d619e2d3-a8f7-4332-9dcd-50a0e06576e6-plugins-config-volume\") pod \"payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt\" (UID: \"d619e2d3-a8f7-4332-9dcd-50a0e06576e6\") " pod="openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt" Jul 29 15:08:38.074554 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:38.074466 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5kcl\" (UniqueName: \"kubernetes.io/projected/d619e2d3-a8f7-4332-9dcd-50a0e06576e6-kube-api-access-j5kcl\") pod \"payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt\" (UID: \"d619e2d3-a8f7-4332-9dcd-50a0e06576e6\") " pod="openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt" Jul 29 15:08:38.074554 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:38.074509 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4eaec18-2534-486e-8d53-e28558cf9be0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp\" (UID: \"f4eaec18-2534-486e-8d53-e28558cf9be0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp" Jul 29 15:08:38.175419 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:38.175354 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4eaec18-2534-486e-8d53-e28558cf9be0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp\" (UID: \"f4eaec18-2534-486e-8d53-e28558cf9be0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp" Jul 29 15:08:38.175575 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:38.175433 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2bhq\" (UniqueName: \"kubernetes.io/projected/f4eaec18-2534-486e-8d53-e28558cf9be0-kube-api-access-n2bhq\") pod \"payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp\" (UID: \"f4eaec18-2534-486e-8d53-e28558cf9be0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp" Jul 29 15:08:38.175575 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:38.175453 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d619e2d3-a8f7-4332-9dcd-50a0e06576e6-plugins-config-volume\") pod \"payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt\" (UID: \"d619e2d3-a8f7-4332-9dcd-50a0e06576e6\") " pod="openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt" Jul 29 15:08:38.175575 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:38.175472 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5kcl\" (UniqueName: \"kubernetes.io/projected/d619e2d3-a8f7-4332-9dcd-50a0e06576e6-kube-api-access-j5kcl\") pod \"payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt\" (UID: \"d619e2d3-a8f7-4332-9dcd-50a0e06576e6\") " pod="openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt" Jul 29 15:08:38.176128 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:38.176109 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4eaec18-2534-486e-8d53-e28558cf9be0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp\" (UID: \"f4eaec18-2534-486e-8d53-e28558cf9be0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp" Jul 29 15:08:38.176205 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:38.176189 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d619e2d3-a8f7-4332-9dcd-50a0e06576e6-plugins-config-volume\") pod \"payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt\" (UID: \"d619e2d3-a8f7-4332-9dcd-50a0e06576e6\") " pod="openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt" Jul 29 15:08:38.188511 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:38.188485 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5kcl\" (UniqueName: \"kubernetes.io/projected/d619e2d3-a8f7-4332-9dcd-50a0e06576e6-kube-api-access-j5kcl\") pod \"payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt\" (UID: \"d619e2d3-a8f7-4332-9dcd-50a0e06576e6\") " pod="openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt" Jul 29 15:08:38.188601 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:38.188582 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2bhq\" (UniqueName: \"kubernetes.io/projected/f4eaec18-2534-486e-8d53-e28558cf9be0-kube-api-access-n2bhq\") pod \"payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp\" (UID: \"f4eaec18-2534-486e-8d53-e28558cf9be0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp" Jul 29 15:08:38.299289 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:38.299264 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt" Jul 29 15:08:38.342335 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:38.342307 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp" Jul 29 15:08:38.457093 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:38.456842 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt"] Jul 29 15:08:38.458151 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:08:38.458122 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd619e2d3_a8f7_4332_9dcd_50a0e06576e6.slice/crio-31ebc9005427f68ba0ced366c9c79cf197f6f63a1cf33ff26e606fe4de2ed0f0 WatchSource:0}: Error finding container 31ebc9005427f68ba0ced366c9c79cf197f6f63a1cf33ff26e606fe4de2ed0f0: Status 404 returned error can't find the container with id 31ebc9005427f68ba0ced366c9c79cf197f6f63a1cf33ff26e606fe4de2ed0f0 Jul 29 15:08:38.701222 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:38.701196 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp"] Jul 29 15:08:38.703190 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:08:38.703164 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4eaec18_2534_486e_8d53_e28558cf9be0.slice/crio-981e675722e783a7da90ad7aa0471b62332272acc62339321625ca9b08bee85a WatchSource:0}: Error finding container 981e675722e783a7da90ad7aa0471b62332272acc62339321625ca9b08bee85a: Status 404 returned error can't find the container with id 981e675722e783a7da90ad7aa0471b62332272acc62339321625ca9b08bee85a Jul 29 15:08:39.107931 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:39.107901 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt" event={"ID":"d619e2d3-a8f7-4332-9dcd-50a0e06576e6","Type":"ContainerStarted","Data":"13825947b75c0659e84d582d3390e152ce8e409d21404cef9589361deaca351e"} Jul 29 15:08:39.108332 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:39.107951 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt" event={"ID":"d619e2d3-a8f7-4332-9dcd-50a0e06576e6","Type":"ContainerStarted","Data":"31ebc9005427f68ba0ced366c9c79cf197f6f63a1cf33ff26e606fe4de2ed0f0"} Jul 29 15:08:39.108332 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:39.107981 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt" Jul 29 15:08:39.109569 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:39.109542 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp" event={"ID":"f4eaec18-2534-486e-8d53-e28558cf9be0","Type":"ContainerStarted","Data":"b5d8eb46f6ecf06dfd5af5ed73242632c8bb1155fa0c002dbe735a2f5eb71a5e"} Jul 29 15:08:39.109694 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:39.109595 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp" Jul 29 15:08:39.109694 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:39.109613 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp" event={"ID":"f4eaec18-2534-486e-8d53-e28558cf9be0","Type":"ContainerStarted","Data":"981e675722e783a7da90ad7aa0471b62332272acc62339321625ca9b08bee85a"} Jul 29 15:08:39.132874 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:39.132836 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt" podStartSLOduration=2.132821662 podStartE2EDuration="2.132821662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:08:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:08:39.131854378 +0000 UTC m=+1995.860003570" watchObservedRunningTime="2026-07-29 15:08:39.132821662 +0000 UTC m=+1995.860970851" Jul 29 15:08:39.153894 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:39.153853 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp" podStartSLOduration=1.153839494 podStartE2EDuration="1.153839494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:08:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:08:39.152046556 +0000 UTC m=+1995.880195748" watchObservedRunningTime="2026-07-29 15:08:39.153839494 +0000 UTC m=+1995.881988727" Jul 29 15:08:50.116002 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:50.115947 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp" podUID="f4eaec18-2534-486e-8d53-e28558cf9be0" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.62:9004: connect: connection refused" Jul 29 15:08:50.116506 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:08:50.116380 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt" Jul 29 15:09:00.115143 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:09:00.115110 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp" Jul 29 15:09:01.888925 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:09:01.888884 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:09:42.148996 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:09:42.148959 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:09:48.597983 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:09:48.597895 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:10:17.891204 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:17.891170 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:10:17.923473 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:17.923440 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt"] Jul 29 15:10:17.923680 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:17.923654 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt" podUID="d619e2d3-a8f7-4332-9dcd-50a0e06576e6" containerName="payload-processing" containerID="cri-o://13825947b75c0659e84d582d3390e152ce8e409d21404cef9589361deaca351e" gracePeriod=30 Jul 29 15:10:17.931277 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:17.931252 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp"] Jul 29 15:10:17.931491 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:17.931468 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp" podUID="f4eaec18-2534-486e-8d53-e28558cf9be0" containerName="payload-pre-processing" containerID="cri-o://b5d8eb46f6ecf06dfd5af5ed73242632c8bb1155fa0c002dbe735a2f5eb71a5e" gracePeriod=30 Jul 29 15:10:17.962997 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:17.962965 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:10:18.027071 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.027040 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:10:18.099710 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.099472 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:10:18.222948 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.222925 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt" Jul 29 15:10:18.225705 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.225683 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp" Jul 29 15:10:18.323555 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.323528 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4eaec18-2534-486e-8d53-e28558cf9be0-plugins-config-volume\") pod \"f4eaec18-2534-486e-8d53-e28558cf9be0\" (UID: \"f4eaec18-2534-486e-8d53-e28558cf9be0\") " Jul 29 15:10:18.323555 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.323558 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j5kcl\" (UniqueName: \"kubernetes.io/projected/d619e2d3-a8f7-4332-9dcd-50a0e06576e6-kube-api-access-j5kcl\") pod \"d619e2d3-a8f7-4332-9dcd-50a0e06576e6\" (UID: \"d619e2d3-a8f7-4332-9dcd-50a0e06576e6\") " Jul 29 15:10:18.323824 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.323610 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2bhq\" (UniqueName: \"kubernetes.io/projected/f4eaec18-2534-486e-8d53-e28558cf9be0-kube-api-access-n2bhq\") pod \"f4eaec18-2534-486e-8d53-e28558cf9be0\" (UID: \"f4eaec18-2534-486e-8d53-e28558cf9be0\") " Jul 29 15:10:18.323824 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.323754 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d619e2d3-a8f7-4332-9dcd-50a0e06576e6-plugins-config-volume\") pod \"d619e2d3-a8f7-4332-9dcd-50a0e06576e6\" (UID: \"d619e2d3-a8f7-4332-9dcd-50a0e06576e6\") " Jul 29 15:10:18.324090 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.324057 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4eaec18-2534-486e-8d53-e28558cf9be0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f4eaec18-2534-486e-8d53-e28558cf9be0" (UID: "f4eaec18-2534-486e-8d53-e28558cf9be0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:10:18.324090 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.324070 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d619e2d3-a8f7-4332-9dcd-50a0e06576e6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d619e2d3-a8f7-4332-9dcd-50a0e06576e6" (UID: "d619e2d3-a8f7-4332-9dcd-50a0e06576e6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:10:18.325661 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.325640 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4eaec18-2534-486e-8d53-e28558cf9be0-kube-api-access-n2bhq" (OuterVolumeSpecName: "kube-api-access-n2bhq") pod "f4eaec18-2534-486e-8d53-e28558cf9be0" (UID: "f4eaec18-2534-486e-8d53-e28558cf9be0"). InnerVolumeSpecName "kube-api-access-n2bhq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:10:18.325775 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.325686 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d619e2d3-a8f7-4332-9dcd-50a0e06576e6-kube-api-access-j5kcl" (OuterVolumeSpecName: "kube-api-access-j5kcl") pod "d619e2d3-a8f7-4332-9dcd-50a0e06576e6" (UID: "d619e2d3-a8f7-4332-9dcd-50a0e06576e6"). InnerVolumeSpecName "kube-api-access-j5kcl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:10:18.425250 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.425189 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4eaec18-2534-486e-8d53-e28558cf9be0-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:10:18.425250 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.425213 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5kcl\" (UniqueName: \"kubernetes.io/projected/d619e2d3-a8f7-4332-9dcd-50a0e06576e6-kube-api-access-j5kcl\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:10:18.425250 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.425223 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2bhq\" (UniqueName: \"kubernetes.io/projected/f4eaec18-2534-486e-8d53-e28558cf9be0-kube-api-access-n2bhq\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:10:18.425250 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.425231 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d619e2d3-a8f7-4332-9dcd-50a0e06576e6-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:10:18.661806 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.661773 2580 generic.go:356] "Generic (PLEG): container finished" podID="d619e2d3-a8f7-4332-9dcd-50a0e06576e6" containerID="13825947b75c0659e84d582d3390e152ce8e409d21404cef9589361deaca351e" exitCode=0 Jul 29 15:10:18.661966 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.661831 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt" Jul 29 15:10:18.661966 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.661859 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt" event={"ID":"d619e2d3-a8f7-4332-9dcd-50a0e06576e6","Type":"ContainerDied","Data":"13825947b75c0659e84d582d3390e152ce8e409d21404cef9589361deaca351e"} Jul 29 15:10:18.661966 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.661938 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt" event={"ID":"d619e2d3-a8f7-4332-9dcd-50a0e06576e6","Type":"ContainerDied","Data":"31ebc9005427f68ba0ced366c9c79cf197f6f63a1cf33ff26e606fe4de2ed0f0"} Jul 29 15:10:18.661966 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.661961 2580 scope.go:117] "RemoveContainer" containerID="13825947b75c0659e84d582d3390e152ce8e409d21404cef9589361deaca351e" Jul 29 15:10:18.663184 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.663152 2580 generic.go:356] "Generic (PLEG): container finished" podID="f4eaec18-2534-486e-8d53-e28558cf9be0" containerID="b5d8eb46f6ecf06dfd5af5ed73242632c8bb1155fa0c002dbe735a2f5eb71a5e" exitCode=0 Jul 29 15:10:18.663294 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.663182 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp" event={"ID":"f4eaec18-2534-486e-8d53-e28558cf9be0","Type":"ContainerDied","Data":"b5d8eb46f6ecf06dfd5af5ed73242632c8bb1155fa0c002dbe735a2f5eb71a5e"} Jul 29 15:10:18.663294 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.663212 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp" Jul 29 15:10:18.663294 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.663232 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp" event={"ID":"f4eaec18-2534-486e-8d53-e28558cf9be0","Type":"ContainerDied","Data":"981e675722e783a7da90ad7aa0471b62332272acc62339321625ca9b08bee85a"} Jul 29 15:10:18.671930 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.671916 2580 scope.go:117] "RemoveContainer" containerID="13825947b75c0659e84d582d3390e152ce8e409d21404cef9589361deaca351e" Jul 29 15:10:18.672176 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:10:18.672159 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13825947b75c0659e84d582d3390e152ce8e409d21404cef9589361deaca351e\": container with ID starting with 13825947b75c0659e84d582d3390e152ce8e409d21404cef9589361deaca351e not found: ID does not exist" containerID="13825947b75c0659e84d582d3390e152ce8e409d21404cef9589361deaca351e" Jul 29 15:10:18.672247 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.672184 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13825947b75c0659e84d582d3390e152ce8e409d21404cef9589361deaca351e"} err="failed to get container status \"13825947b75c0659e84d582d3390e152ce8e409d21404cef9589361deaca351e\": rpc error: code = NotFound desc = could not find container \"13825947b75c0659e84d582d3390e152ce8e409d21404cef9589361deaca351e\": container with ID starting with 13825947b75c0659e84d582d3390e152ce8e409d21404cef9589361deaca351e not found: ID does not exist" Jul 29 15:10:18.672247 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.672202 2580 scope.go:117] "RemoveContainer" containerID="b5d8eb46f6ecf06dfd5af5ed73242632c8bb1155fa0c002dbe735a2f5eb71a5e" Jul 29 15:10:18.679992 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.679977 2580 scope.go:117] "RemoveContainer" containerID="b5d8eb46f6ecf06dfd5af5ed73242632c8bb1155fa0c002dbe735a2f5eb71a5e" Jul 29 15:10:18.680234 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:10:18.680217 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5d8eb46f6ecf06dfd5af5ed73242632c8bb1155fa0c002dbe735a2f5eb71a5e\": container with ID starting with b5d8eb46f6ecf06dfd5af5ed73242632c8bb1155fa0c002dbe735a2f5eb71a5e not found: ID does not exist" containerID="b5d8eb46f6ecf06dfd5af5ed73242632c8bb1155fa0c002dbe735a2f5eb71a5e" Jul 29 15:10:18.680293 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.680238 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5d8eb46f6ecf06dfd5af5ed73242632c8bb1155fa0c002dbe735a2f5eb71a5e"} err="failed to get container status \"b5d8eb46f6ecf06dfd5af5ed73242632c8bb1155fa0c002dbe735a2f5eb71a5e\": rpc error: code = NotFound desc = could not find container \"b5d8eb46f6ecf06dfd5af5ed73242632c8bb1155fa0c002dbe735a2f5eb71a5e\": container with ID starting with b5d8eb46f6ecf06dfd5af5ed73242632c8bb1155fa0c002dbe735a2f5eb71a5e not found: ID does not exist" Jul 29 15:10:18.701052 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.701029 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp"] Jul 29 15:10:18.714474 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.714449 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-a385d8ab-7b79b9b6f6-7c8rp"] Jul 29 15:10:18.737519 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.737495 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt"] Jul 29 15:10:18.746142 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:18.746113 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-a385d8ab-76c6744f78-fhgkt"] Jul 29 15:10:19.818104 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:19.818069 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d619e2d3-a8f7-4332-9dcd-50a0e06576e6" path="/var/lib/kubelet/pods/d619e2d3-a8f7-4332-9dcd-50a0e06576e6/volumes" Jul 29 15:10:19.818594 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:19.818571 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4eaec18-2534-486e-8d53-e28558cf9be0" path="/var/lib/kubelet/pods/f4eaec18-2534-486e-8d53-e28558cf9be0/volumes" Jul 29 15:10:21.574830 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:21.574784 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:10:23.890916 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:23.890889 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 15:10:23.895422 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:23.895400 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 15:10:26.606797 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:10:26.606758 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:02.820158 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:02.820113 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:03.899606 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:03.899570 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:09.490909 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:09.490869 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:15.472775 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:15.472667 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:15.574627 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:15.574592 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:15.683223 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:15.683192 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:15.780047 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:15.780012 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:16.033511 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:16.033410 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:17.162945 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:17.162904 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:17.258284 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:17.258249 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:17.357244 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:17.357208 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:18.183127 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:18.183077 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:18.280364 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:18.280336 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:18.402618 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:18.402593 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:19.883081 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:19.883049 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:25.722187 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:25.722149 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:27.305558 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:27.305524 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:27.359970 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:27.359942 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:34.138131 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:34.138093 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:35.769150 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:35.769112 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:43.470169 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:43.470131 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:43.868185 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:43.868151 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:11:59.847216 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:11:59.847173 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:04.930221 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:04.930180 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:11.260220 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:11.260186 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:11.353961 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:11.353932 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:11.640715 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:11.640633 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:12.010621 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:12.010583 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:12.312955 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:12.312874 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:14.926342 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:14.926298 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:15.032278 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:15.032241 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:15.126687 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:15.126645 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:15.610818 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:15.610782 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:15.784543 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:15.784513 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:15.884574 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:15.884502 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:15.919264 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:15.919237 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj"] Jul 29 15:12:15.919617 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:15.919604 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4eaec18-2534-486e-8d53-e28558cf9be0" containerName="payload-pre-processing" Jul 29 15:12:15.919665 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:15.919619 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4eaec18-2534-486e-8d53-e28558cf9be0" containerName="payload-pre-processing" Jul 29 15:12:15.919665 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:15.919632 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d619e2d3-a8f7-4332-9dcd-50a0e06576e6" containerName="payload-processing" Jul 29 15:12:15.919665 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:15.919638 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d619e2d3-a8f7-4332-9dcd-50a0e06576e6" containerName="payload-processing" Jul 29 15:12:15.919797 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:15.919700 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d619e2d3-a8f7-4332-9dcd-50a0e06576e6" containerName="payload-processing" Jul 29 15:12:15.919797 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:15.919708 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4eaec18-2534-486e-8d53-e28558cf9be0" containerName="payload-pre-processing" Jul 29 15:12:15.922561 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:15.922544 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj" Jul 29 15:12:15.924892 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:15.924870 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-5ef0d2\"" Jul 29 15:12:15.924987 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:15.924896 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-5ef0d2-dockercfg-64tv7\"" Jul 29 15:12:15.935856 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:15.935834 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj"] Jul 29 15:12:15.965280 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:15.965255 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n"] Jul 29 15:12:15.968554 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:15.968537 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n" Jul 29 15:12:15.981437 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:15.981418 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n"] Jul 29 15:12:16.074495 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:16.074472 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17585554-128c-400f-afa0-cbf3c1a93983-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj\" (UID: \"17585554-128c-400f-afa0-cbf3c1a93983\") " pod="openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj" Jul 29 15:12:16.074652 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:16.074502 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf85726a-28d0-4eb7-9864-b29a9cabb68c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n\" (UID: \"bf85726a-28d0-4eb7-9864-b29a9cabb68c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n" Jul 29 15:12:16.074652 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:16.074543 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5zk6\" (UniqueName: \"kubernetes.io/projected/bf85726a-28d0-4eb7-9864-b29a9cabb68c-kube-api-access-v5zk6\") pod \"payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n\" (UID: \"bf85726a-28d0-4eb7-9864-b29a9cabb68c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n" Jul 29 15:12:16.074652 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:16.074631 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfd7l\" (UniqueName: \"kubernetes.io/projected/17585554-128c-400f-afa0-cbf3c1a93983-kube-api-access-sfd7l\") pod \"payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj\" (UID: \"17585554-128c-400f-afa0-cbf3c1a93983\") " pod="openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj" Jul 29 15:12:16.175203 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:16.175130 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5zk6\" (UniqueName: \"kubernetes.io/projected/bf85726a-28d0-4eb7-9864-b29a9cabb68c-kube-api-access-v5zk6\") pod \"payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n\" (UID: \"bf85726a-28d0-4eb7-9864-b29a9cabb68c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n" Jul 29 15:12:16.175203 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:16.175180 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfd7l\" (UniqueName: \"kubernetes.io/projected/17585554-128c-400f-afa0-cbf3c1a93983-kube-api-access-sfd7l\") pod \"payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj\" (UID: \"17585554-128c-400f-afa0-cbf3c1a93983\") " pod="openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj" Jul 29 15:12:16.175395 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:16.175239 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17585554-128c-400f-afa0-cbf3c1a93983-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj\" (UID: \"17585554-128c-400f-afa0-cbf3c1a93983\") " pod="openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj" Jul 29 15:12:16.175395 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:16.175267 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf85726a-28d0-4eb7-9864-b29a9cabb68c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n\" (UID: \"bf85726a-28d0-4eb7-9864-b29a9cabb68c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n" Jul 29 15:12:16.176008 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:16.175984 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf85726a-28d0-4eb7-9864-b29a9cabb68c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n\" (UID: \"bf85726a-28d0-4eb7-9864-b29a9cabb68c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n" Jul 29 15:12:16.176097 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:16.176053 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17585554-128c-400f-afa0-cbf3c1a93983-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj\" (UID: \"17585554-128c-400f-afa0-cbf3c1a93983\") " pod="openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj" Jul 29 15:12:16.182830 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:16.182803 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfd7l\" (UniqueName: \"kubernetes.io/projected/17585554-128c-400f-afa0-cbf3c1a93983-kube-api-access-sfd7l\") pod \"payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj\" (UID: \"17585554-128c-400f-afa0-cbf3c1a93983\") " pod="openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj" Jul 29 15:12:16.183077 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:16.183061 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5zk6\" (UniqueName: \"kubernetes.io/projected/bf85726a-28d0-4eb7-9864-b29a9cabb68c-kube-api-access-v5zk6\") pod \"payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n\" (UID: \"bf85726a-28d0-4eb7-9864-b29a9cabb68c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n" Jul 29 15:12:16.232951 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:16.232926 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj" Jul 29 15:12:16.278840 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:16.278813 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n" Jul 29 15:12:16.583698 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:16.583671 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj"] Jul 29 15:12:16.586069 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:12:16.586042 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17585554_128c_400f_afa0_cbf3c1a93983.slice/crio-f68991695f2c16b4d9042f111615de7e784729e122fe032eff0accdc91119a09 WatchSource:0}: Error finding container f68991695f2c16b4d9042f111615de7e784729e122fe032eff0accdc91119a09: Status 404 returned error can't find the container with id f68991695f2c16b4d9042f111615de7e784729e122fe032eff0accdc91119a09 Jul 29 15:12:16.629746 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:16.629709 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n"] Jul 29 15:12:16.630916 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:12:16.630893 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf85726a_28d0_4eb7_9864_b29a9cabb68c.slice/crio-246c101e7d55bb6c053819af4953b86a69869f892dbf75321e629683ecbd602e WatchSource:0}: Error finding container 246c101e7d55bb6c053819af4953b86a69869f892dbf75321e629683ecbd602e: Status 404 returned error can't find the container with id 246c101e7d55bb6c053819af4953b86a69869f892dbf75321e629683ecbd602e Jul 29 15:12:17.309430 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:17.309382 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n" event={"ID":"bf85726a-28d0-4eb7-9864-b29a9cabb68c","Type":"ContainerStarted","Data":"0144fbb2db6776f2ba9a4bc0eac83a66f36a538aea3b303f2b923b6dffd335c1"} Jul 29 15:12:17.309868 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:17.309492 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n" event={"ID":"bf85726a-28d0-4eb7-9864-b29a9cabb68c","Type":"ContainerStarted","Data":"246c101e7d55bb6c053819af4953b86a69869f892dbf75321e629683ecbd602e"} Jul 29 15:12:17.309868 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:17.309524 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n" Jul 29 15:12:17.310909 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:17.310888 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj" event={"ID":"17585554-128c-400f-afa0-cbf3c1a93983","Type":"ContainerStarted","Data":"db6985845d34bb0fc9291dc496ac92858b5b27ae65bafb04a83db45a963bf9e5"} Jul 29 15:12:17.310963 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:17.310931 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj" event={"ID":"17585554-128c-400f-afa0-cbf3c1a93983","Type":"ContainerStarted","Data":"f68991695f2c16b4d9042f111615de7e784729e122fe032eff0accdc91119a09"} Jul 29 15:12:17.311004 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:17.310964 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj" Jul 29 15:12:17.330320 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:17.330276 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n" podStartSLOduration=2.330263173 podStartE2EDuration="2.330263173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:12:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:17.328148548 +0000 UTC m=+2214.056297738" watchObservedRunningTime="2026-07-29 15:12:17.330263173 +0000 UTC m=+2214.058412362" Jul 29 15:12:17.352088 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:17.352050 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj" podStartSLOduration=2.35203745 podStartE2EDuration="2.35203745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:12:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:17.349972827 +0000 UTC m=+2214.078122018" watchObservedRunningTime="2026-07-29 15:12:17.35203745 +0000 UTC m=+2214.080186692" Jul 29 15:12:28.318585 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:28.318553 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj" Jul 29 15:12:28.319033 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:28.318751 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n" Jul 29 15:12:40.048989 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:40.048929 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:40.508290 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:40.508253 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:40.791639 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:40.791549 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:41.201073 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:41.200969 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:44.093412 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:44.093374 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:44.194403 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:44.194366 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:44.353489 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:44.353388 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:44.915028 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:44.914995 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:45.026055 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.026021 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:45.026944 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.026925 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd"] Jul 29 15:12:45.030927 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.030903 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd" Jul 29 15:12:45.032966 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.032925 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-80fc9d-dockercfg-4plbn\"" Jul 29 15:12:45.033159 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.033140 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-80fc9d\"" Jul 29 15:12:45.039602 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.039578 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd"] Jul 29 15:12:45.058419 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.058384 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l"] Jul 29 15:12:45.061815 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.061794 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l" Jul 29 15:12:45.071658 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.071633 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l"] Jul 29 15:12:45.114229 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.114184 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:12:45.134375 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.134339 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kngt4\" (UniqueName: \"kubernetes.io/projected/9e91b19c-6d99-4482-a028-c287f1a01d89-kube-api-access-kngt4\") pod \"payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l\" (UID: \"9e91b19c-6d99-4482-a028-c287f1a01d89\") " pod="openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l" Jul 29 15:12:45.134375 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.134383 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe003141-6a4c-4351-ba88-132a5c6cd083-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd\" (UID: \"fe003141-6a4c-4351-ba88-132a5c6cd083\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd" Jul 29 15:12:45.134594 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.134413 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9bxk\" (UniqueName: \"kubernetes.io/projected/fe003141-6a4c-4351-ba88-132a5c6cd083-kube-api-access-j9bxk\") pod \"payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd\" (UID: \"fe003141-6a4c-4351-ba88-132a5c6cd083\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd" Jul 29 15:12:45.134594 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.134517 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e91b19c-6d99-4482-a028-c287f1a01d89-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l\" (UID: \"9e91b19c-6d99-4482-a028-c287f1a01d89\") " pod="openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l" Jul 29 15:12:45.235524 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.235431 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kngt4\" (UniqueName: \"kubernetes.io/projected/9e91b19c-6d99-4482-a028-c287f1a01d89-kube-api-access-kngt4\") pod \"payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l\" (UID: \"9e91b19c-6d99-4482-a028-c287f1a01d89\") " pod="openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l" Jul 29 15:12:45.235524 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.235478 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe003141-6a4c-4351-ba88-132a5c6cd083-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd\" (UID: \"fe003141-6a4c-4351-ba88-132a5c6cd083\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd" Jul 29 15:12:45.235767 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.235531 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9bxk\" (UniqueName: \"kubernetes.io/projected/fe003141-6a4c-4351-ba88-132a5c6cd083-kube-api-access-j9bxk\") pod \"payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd\" (UID: \"fe003141-6a4c-4351-ba88-132a5c6cd083\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd" Jul 29 15:12:45.235767 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.235574 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e91b19c-6d99-4482-a028-c287f1a01d89-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l\" (UID: \"9e91b19c-6d99-4482-a028-c287f1a01d89\") " pod="openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l" Jul 29 15:12:45.236295 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.236278 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe003141-6a4c-4351-ba88-132a5c6cd083-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd\" (UID: \"fe003141-6a4c-4351-ba88-132a5c6cd083\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd" Jul 29 15:12:45.236394 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.236377 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e91b19c-6d99-4482-a028-c287f1a01d89-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l\" (UID: \"9e91b19c-6d99-4482-a028-c287f1a01d89\") " pod="openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l" Jul 29 15:12:45.243527 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.243498 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kngt4\" (UniqueName: \"kubernetes.io/projected/9e91b19c-6d99-4482-a028-c287f1a01d89-kube-api-access-kngt4\") pod \"payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l\" (UID: \"9e91b19c-6d99-4482-a028-c287f1a01d89\") " pod="openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l" Jul 29 15:12:45.243646 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.243544 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9bxk\" (UniqueName: \"kubernetes.io/projected/fe003141-6a4c-4351-ba88-132a5c6cd083-kube-api-access-j9bxk\") pod \"payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd\" (UID: \"fe003141-6a4c-4351-ba88-132a5c6cd083\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd" Jul 29 15:12:45.343775 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.343643 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd" Jul 29 15:12:45.373225 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.373189 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l" Jul 29 15:12:45.505488 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.505459 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd"] Jul 29 15:12:45.507739 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:12:45.507691 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe003141_6a4c_4351_ba88_132a5c6cd083.slice/crio-e35d6ee5c7039609bb8b9b65e32f8c10b02364a133b99b3838b1791db50f1320 WatchSource:0}: Error finding container e35d6ee5c7039609bb8b9b65e32f8c10b02364a133b99b3838b1791db50f1320: Status 404 returned error can't find the container with id e35d6ee5c7039609bb8b9b65e32f8c10b02364a133b99b3838b1791db50f1320 Jul 29 15:12:45.735681 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:45.735657 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l"] Jul 29 15:12:45.737484 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:12:45.737459 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e91b19c_6d99_4482_a028_c287f1a01d89.slice/crio-08288c88cef8dd897d002485b96a233dddf50cf6b3133fe60d215d3572344ead WatchSource:0}: Error finding container 08288c88cef8dd897d002485b96a233dddf50cf6b3133fe60d215d3572344ead: Status 404 returned error can't find the container with id 08288c88cef8dd897d002485b96a233dddf50cf6b3133fe60d215d3572344ead Jul 29 15:12:46.477332 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:46.477291 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l" event={"ID":"9e91b19c-6d99-4482-a028-c287f1a01d89","Type":"ContainerStarted","Data":"4dce553475babe4904841c3560aaeed48638cd832b0bcdfc8ab817f42b7423f5"} Jul 29 15:12:46.477820 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:46.477348 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l" event={"ID":"9e91b19c-6d99-4482-a028-c287f1a01d89","Type":"ContainerStarted","Data":"08288c88cef8dd897d002485b96a233dddf50cf6b3133fe60d215d3572344ead"} Jul 29 15:12:46.477820 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:46.477377 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l" Jul 29 15:12:46.478891 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:46.478868 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd" event={"ID":"fe003141-6a4c-4351-ba88-132a5c6cd083","Type":"ContainerStarted","Data":"1c7bd918d5b556361e05b6c6aba13d3bcd95b445096b55644cb4cddf5bcc3057"} Jul 29 15:12:46.479012 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:46.478911 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd" event={"ID":"fe003141-6a4c-4351-ba88-132a5c6cd083","Type":"ContainerStarted","Data":"e35d6ee5c7039609bb8b9b65e32f8c10b02364a133b99b3838b1791db50f1320"} Jul 29 15:12:46.479012 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:46.479000 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd" Jul 29 15:12:46.494699 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:46.494641 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l" podStartSLOduration=1.494626124 podStartE2EDuration="1.494626124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:12:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:46.491932001 +0000 UTC m=+2243.220081193" watchObservedRunningTime="2026-07-29 15:12:46.494626124 +0000 UTC m=+2243.222775313" Jul 29 15:12:46.507478 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:46.507429 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd" podStartSLOduration=1.507417909 podStartE2EDuration="1.507417909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:12:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:46.506423349 +0000 UTC m=+2243.234572547" watchObservedRunningTime="2026-07-29 15:12:46.507417909 +0000 UTC m=+2243.235567098" Jul 29 15:12:57.486493 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:57.486457 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l" Jul 29 15:12:57.487003 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:12:57.486571 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd" Jul 29 15:13:09.160273 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:13:09.160234 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:13:15.253167 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:13:15.253130 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:13:56.903829 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:13:56.903760 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:14:01.301979 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:01.301920 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:14:01.358617 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:01.358577 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:14:21.075818 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:21.075756 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:14:33.497344 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:33.497298 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:14:39.456957 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:39.456921 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:14:40.371585 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:40.371549 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:14:40.490182 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:40.490151 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ef0d2-6tdlt"] Jul 29 15:14:40.493684 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:40.493666 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ef0d2-6tdlt" Jul 29 15:14:40.495685 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:40.495663 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mmt6p\"" Jul 29 15:14:40.495857 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:40.495665 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 29 15:14:40.500024 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:40.499999 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ef0d2-6tdlt"] Jul 29 15:14:40.593555 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:40.593519 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3790f86f-2fcf-4ed9-ab12-36267e6c2f04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-5ef0d2-6tdlt\" (UID: \"3790f86f-2fcf-4ed9-ab12-36267e6c2f04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ef0d2-6tdlt" Jul 29 15:14:40.694101 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:40.694004 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3790f86f-2fcf-4ed9-ab12-36267e6c2f04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-5ef0d2-6tdlt\" (UID: \"3790f86f-2fcf-4ed9-ab12-36267e6c2f04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ef0d2-6tdlt" Jul 29 15:14:40.694693 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:40.694665 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3790f86f-2fcf-4ed9-ab12-36267e6c2f04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-5ef0d2-6tdlt\" (UID: \"3790f86f-2fcf-4ed9-ab12-36267e6c2f04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ef0d2-6tdlt" Jul 29 15:14:40.805909 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:40.805875 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ef0d2-6tdlt" Jul 29 15:14:40.944141 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:40.944082 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ef0d2-6tdlt"] Jul 29 15:14:40.945166 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:14:40.945140 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3790f86f_2fcf_4ed9_ab12_36267e6c2f04.slice/crio-e3b0543d9032b9aa00ab6c50dd3f4f3bc623b3c4ff601af310151f8689968e83 WatchSource:0}: Error finding container e3b0543d9032b9aa00ab6c50dd3f4f3bc623b3c4ff601af310151f8689968e83: Status 404 returned error can't find the container with id e3b0543d9032b9aa00ab6c50dd3f4f3bc623b3c4ff601af310151f8689968e83 Jul 29 15:14:41.120067 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:41.120035 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ef0d2-6tdlt" event={"ID":"3790f86f-2fcf-4ed9-ab12-36267e6c2f04","Type":"ContainerStarted","Data":"e3b0543d9032b9aa00ab6c50dd3f4f3bc623b3c4ff601af310151f8689968e83"} Jul 29 15:14:42.127251 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:42.127207 2580 generic.go:356] "Generic (PLEG): container finished" podID="3790f86f-2fcf-4ed9-ab12-36267e6c2f04" containerID="8f8d5f4937ce37023871a5665eeafe5279acbcccbf20ab6426080aed5d4d1bf6" exitCode=0 Jul 29 15:14:42.127251 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:42.127249 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ef0d2-6tdlt" event={"ID":"3790f86f-2fcf-4ed9-ab12-36267e6c2f04","Type":"ContainerDied","Data":"8f8d5f4937ce37023871a5665eeafe5279acbcccbf20ab6426080aed5d4d1bf6"} Jul 29 15:14:43.267379 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:43.267355 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ef0d2-6tdlt" Jul 29 15:14:43.316557 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:43.316526 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3790f86f-2fcf-4ed9-ab12-36267e6c2f04-maas-api-service-ca\") pod \"3790f86f-2fcf-4ed9-ab12-36267e6c2f04\" (UID: \"3790f86f-2fcf-4ed9-ab12-36267e6c2f04\") " Jul 29 15:14:43.316943 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:43.316924 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3790f86f-2fcf-4ed9-ab12-36267e6c2f04-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3790f86f-2fcf-4ed9-ab12-36267e6c2f04" (UID: "3790f86f-2fcf-4ed9-ab12-36267e6c2f04"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:14:43.417255 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:43.417171 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3790f86f-2fcf-4ed9-ab12-36267e6c2f04-maas-api-service-ca\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:14:44.139915 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:44.139826 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ef0d2-6tdlt" Jul 29 15:14:44.139915 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:44.139851 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ef0d2-6tdlt" event={"ID":"3790f86f-2fcf-4ed9-ab12-36267e6c2f04","Type":"ContainerDied","Data":"e3b0543d9032b9aa00ab6c50dd3f4f3bc623b3c4ff601af310151f8689968e83"} Jul 29 15:14:44.139915 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:44.139906 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3b0543d9032b9aa00ab6c50dd3f4f3bc623b3c4ff601af310151f8689968e83" Jul 29 15:14:51.129171 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.129134 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:14:51.158496 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.158460 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj"] Jul 29 15:14:51.158759 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.158713 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj" podUID="17585554-128c-400f-afa0-cbf3c1a93983" containerName="payload-processing" containerID="cri-o://db6985845d34bb0fc9291dc496ac92858b5b27ae65bafb04a83db45a963bf9e5" gracePeriod=30 Jul 29 15:14:51.165306 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.165270 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n"] Jul 29 15:14:51.165525 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.165493 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n" podUID="bf85726a-28d0-4eb7-9864-b29a9cabb68c" containerName="payload-pre-processing" containerID="cri-o://0144fbb2db6776f2ba9a4bc0eac83a66f36a538aea3b303f2b923b6dffd335c1" gracePeriod=30 Jul 29 15:14:51.202267 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.202236 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:14:51.265075 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.264979 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:14:51.342356 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.342300 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ef0d2-6tdlt"] Jul 29 15:14:51.345360 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.345336 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-5ef0d2-6tdlt"] Jul 29 15:14:51.350254 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.350226 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:14:51.457443 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.457421 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj" Jul 29 15:14:51.460786 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.460763 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n" Jul 29 15:14:51.492021 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.491988 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfd7l\" (UniqueName: \"kubernetes.io/projected/17585554-128c-400f-afa0-cbf3c1a93983-kube-api-access-sfd7l\") pod \"17585554-128c-400f-afa0-cbf3c1a93983\" (UID: \"17585554-128c-400f-afa0-cbf3c1a93983\") " Jul 29 15:14:51.492163 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.492077 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf85726a-28d0-4eb7-9864-b29a9cabb68c-plugins-config-volume\") pod \"bf85726a-28d0-4eb7-9864-b29a9cabb68c\" (UID: \"bf85726a-28d0-4eb7-9864-b29a9cabb68c\") " Jul 29 15:14:51.492163 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.492120 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17585554-128c-400f-afa0-cbf3c1a93983-plugins-config-volume\") pod \"17585554-128c-400f-afa0-cbf3c1a93983\" (UID: \"17585554-128c-400f-afa0-cbf3c1a93983\") " Jul 29 15:14:51.492287 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.492180 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5zk6\" (UniqueName: \"kubernetes.io/projected/bf85726a-28d0-4eb7-9864-b29a9cabb68c-kube-api-access-v5zk6\") pod \"bf85726a-28d0-4eb7-9864-b29a9cabb68c\" (UID: \"bf85726a-28d0-4eb7-9864-b29a9cabb68c\") " Jul 29 15:14:51.492661 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.492490 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf85726a-28d0-4eb7-9864-b29a9cabb68c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bf85726a-28d0-4eb7-9864-b29a9cabb68c" (UID: "bf85726a-28d0-4eb7-9864-b29a9cabb68c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:14:51.492868 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.492687 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17585554-128c-400f-afa0-cbf3c1a93983-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "17585554-128c-400f-afa0-cbf3c1a93983" (UID: "17585554-128c-400f-afa0-cbf3c1a93983"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:14:51.495150 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.495111 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf85726a-28d0-4eb7-9864-b29a9cabb68c-kube-api-access-v5zk6" (OuterVolumeSpecName: "kube-api-access-v5zk6") pod "bf85726a-28d0-4eb7-9864-b29a9cabb68c" (UID: "bf85726a-28d0-4eb7-9864-b29a9cabb68c"). InnerVolumeSpecName "kube-api-access-v5zk6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:14:51.497369 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.496458 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17585554-128c-400f-afa0-cbf3c1a93983-kube-api-access-sfd7l" (OuterVolumeSpecName: "kube-api-access-sfd7l") pod "17585554-128c-400f-afa0-cbf3c1a93983" (UID: "17585554-128c-400f-afa0-cbf3c1a93983"). InnerVolumeSpecName "kube-api-access-sfd7l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:14:51.593217 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.593185 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfd7l\" (UniqueName: \"kubernetes.io/projected/17585554-128c-400f-afa0-cbf3c1a93983-kube-api-access-sfd7l\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:14:51.593217 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.593214 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf85726a-28d0-4eb7-9864-b29a9cabb68c-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:14:51.593420 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.593226 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17585554-128c-400f-afa0-cbf3c1a93983-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:14:51.593420 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.593238 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5zk6\" (UniqueName: \"kubernetes.io/projected/bf85726a-28d0-4eb7-9864-b29a9cabb68c-kube-api-access-v5zk6\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:14:51.723863 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.723748 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:14:51.817650 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:51.817617 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3790f86f-2fcf-4ed9-ab12-36267e6c2f04" path="/var/lib/kubelet/pods/3790f86f-2fcf-4ed9-ab12-36267e6c2f04/volumes" Jul 29 15:14:52.188708 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:52.188666 2580 generic.go:356] "Generic (PLEG): container finished" podID="17585554-128c-400f-afa0-cbf3c1a93983" containerID="db6985845d34bb0fc9291dc496ac92858b5b27ae65bafb04a83db45a963bf9e5" exitCode=0 Jul 29 15:14:52.189183 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:52.188738 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj" Jul 29 15:14:52.189183 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:52.188751 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj" event={"ID":"17585554-128c-400f-afa0-cbf3c1a93983","Type":"ContainerDied","Data":"db6985845d34bb0fc9291dc496ac92858b5b27ae65bafb04a83db45a963bf9e5"} Jul 29 15:14:52.189183 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:52.188845 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj" event={"ID":"17585554-128c-400f-afa0-cbf3c1a93983","Type":"ContainerDied","Data":"f68991695f2c16b4d9042f111615de7e784729e122fe032eff0accdc91119a09"} Jul 29 15:14:52.189183 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:52.188872 2580 scope.go:117] "RemoveContainer" containerID="db6985845d34bb0fc9291dc496ac92858b5b27ae65bafb04a83db45a963bf9e5" Jul 29 15:14:52.190276 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:52.190250 2580 generic.go:356] "Generic (PLEG): container finished" podID="bf85726a-28d0-4eb7-9864-b29a9cabb68c" containerID="0144fbb2db6776f2ba9a4bc0eac83a66f36a538aea3b303f2b923b6dffd335c1" exitCode=0 Jul 29 15:14:52.190474 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:52.190315 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n" Jul 29 15:14:52.190627 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:52.190325 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n" event={"ID":"bf85726a-28d0-4eb7-9864-b29a9cabb68c","Type":"ContainerDied","Data":"0144fbb2db6776f2ba9a4bc0eac83a66f36a538aea3b303f2b923b6dffd335c1"} Jul 29 15:14:52.190756 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:52.190647 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n" event={"ID":"bf85726a-28d0-4eb7-9864-b29a9cabb68c","Type":"ContainerDied","Data":"246c101e7d55bb6c053819af4953b86a69869f892dbf75321e629683ecbd602e"} Jul 29 15:14:52.201037 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:52.200991 2580 scope.go:117] "RemoveContainer" containerID="db6985845d34bb0fc9291dc496ac92858b5b27ae65bafb04a83db45a963bf9e5" Jul 29 15:14:52.201498 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:14:52.201472 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db6985845d34bb0fc9291dc496ac92858b5b27ae65bafb04a83db45a963bf9e5\": container with ID starting with db6985845d34bb0fc9291dc496ac92858b5b27ae65bafb04a83db45a963bf9e5 not found: ID does not exist" containerID="db6985845d34bb0fc9291dc496ac92858b5b27ae65bafb04a83db45a963bf9e5" Jul 29 15:14:52.201567 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:52.201508 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db6985845d34bb0fc9291dc496ac92858b5b27ae65bafb04a83db45a963bf9e5"} err="failed to get container status \"db6985845d34bb0fc9291dc496ac92858b5b27ae65bafb04a83db45a963bf9e5\": rpc error: code = NotFound desc = could not find container \"db6985845d34bb0fc9291dc496ac92858b5b27ae65bafb04a83db45a963bf9e5\": container with ID starting with db6985845d34bb0fc9291dc496ac92858b5b27ae65bafb04a83db45a963bf9e5 not found: ID does not exist" Jul 29 15:14:52.201567 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:52.201530 2580 scope.go:117] "RemoveContainer" containerID="0144fbb2db6776f2ba9a4bc0eac83a66f36a538aea3b303f2b923b6dffd335c1" Jul 29 15:14:52.209259 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:52.209238 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n"] Jul 29 15:14:52.210175 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:52.210096 2580 scope.go:117] "RemoveContainer" containerID="0144fbb2db6776f2ba9a4bc0eac83a66f36a538aea3b303f2b923b6dffd335c1" Jul 29 15:14:52.210422 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:14:52.210397 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0144fbb2db6776f2ba9a4bc0eac83a66f36a538aea3b303f2b923b6dffd335c1\": container with ID starting with 0144fbb2db6776f2ba9a4bc0eac83a66f36a538aea3b303f2b923b6dffd335c1 not found: ID does not exist" containerID="0144fbb2db6776f2ba9a4bc0eac83a66f36a538aea3b303f2b923b6dffd335c1" Jul 29 15:14:52.210570 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:52.210542 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0144fbb2db6776f2ba9a4bc0eac83a66f36a538aea3b303f2b923b6dffd335c1"} err="failed to get container status \"0144fbb2db6776f2ba9a4bc0eac83a66f36a538aea3b303f2b923b6dffd335c1\": rpc error: code = NotFound desc = could not find container \"0144fbb2db6776f2ba9a4bc0eac83a66f36a538aea3b303f2b923b6dffd335c1\": container with ID starting with 0144fbb2db6776f2ba9a4bc0eac83a66f36a538aea3b303f2b923b6dffd335c1 not found: ID does not exist" Jul 29 15:14:52.212989 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:52.212968 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-5ef0d2-55b8dd4484-fmd4n"] Jul 29 15:14:52.223876 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:52.223854 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj"] Jul 29 15:14:52.230401 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:52.230381 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-5ef0d2-675b764f98-wrzgj"] Jul 29 15:14:53.817966 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:53.817930 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17585554-128c-400f-afa0-cbf3c1a93983" path="/var/lib/kubelet/pods/17585554-128c-400f-afa0-cbf3c1a93983/volumes" Jul 29 15:14:53.818355 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:53.818278 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf85726a-28d0-4eb7-9864-b29a9cabb68c" path="/var/lib/kubelet/pods/bf85726a-28d0-4eb7-9864-b29a9cabb68c/volumes" Jul 29 15:14:57.641261 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:57.641223 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:14:57.716290 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:57.716256 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:14:58.001147 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:58.001095 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:14:58.239511 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:58.239467 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:14:58.340911 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:58.340824 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80fc9d-9qjc6"] Jul 29 15:14:58.341287 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:58.341274 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf85726a-28d0-4eb7-9864-b29a9cabb68c" containerName="payload-pre-processing" Jul 29 15:14:58.341339 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:58.341289 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf85726a-28d0-4eb7-9864-b29a9cabb68c" containerName="payload-pre-processing" Jul 29 15:14:58.341339 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:58.341301 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17585554-128c-400f-afa0-cbf3c1a93983" containerName="payload-processing" Jul 29 15:14:58.341339 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:58.341307 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="17585554-128c-400f-afa0-cbf3c1a93983" containerName="payload-processing" Jul 29 15:14:58.341339 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:58.341317 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3790f86f-2fcf-4ed9-ab12-36267e6c2f04" containerName="revoke-keys" Jul 29 15:14:58.341339 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:58.341322 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3790f86f-2fcf-4ed9-ab12-36267e6c2f04" containerName="revoke-keys" Jul 29 15:14:58.341493 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:58.341387 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf85726a-28d0-4eb7-9864-b29a9cabb68c" containerName="payload-pre-processing" Jul 29 15:14:58.341493 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:58.341394 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="17585554-128c-400f-afa0-cbf3c1a93983" containerName="payload-processing" Jul 29 15:14:58.341493 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:58.341402 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3790f86f-2fcf-4ed9-ab12-36267e6c2f04" containerName="revoke-keys" Jul 29 15:14:58.345863 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:58.345843 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80fc9d-9qjc6" Jul 29 15:14:58.347826 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:58.347776 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mmt6p\"" Jul 29 15:14:58.347980 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:58.347788 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 29 15:14:58.354676 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:58.354200 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80fc9d-9qjc6"] Jul 29 15:14:58.460705 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:58.460671 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fed510e-03c0-4030-80ba-f3c493601374-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-80fc9d-9qjc6\" (UID: \"9fed510e-03c0-4030-80ba-f3c493601374\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80fc9d-9qjc6" Jul 29 15:14:58.561573 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:58.561524 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fed510e-03c0-4030-80ba-f3c493601374-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-80fc9d-9qjc6\" (UID: \"9fed510e-03c0-4030-80ba-f3c493601374\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80fc9d-9qjc6" Jul 29 15:14:58.562166 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:58.562146 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fed510e-03c0-4030-80ba-f3c493601374-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-80fc9d-9qjc6\" (UID: \"9fed510e-03c0-4030-80ba-f3c493601374\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80fc9d-9qjc6" Jul 29 15:14:58.660447 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:58.660357 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80fc9d-9qjc6" Jul 29 15:14:58.804546 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:58.804516 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80fc9d-9qjc6"] Jul 29 15:14:58.806758 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:14:58.806691 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fed510e_03c0_4030_80ba_f3c493601374.slice/crio-12ed4119b3d4a8ef45bcd762fc72e17844c14aeb5f6362026d7aeed1d7e73ac0 WatchSource:0}: Error finding container 12ed4119b3d4a8ef45bcd762fc72e17844c14aeb5f6362026d7aeed1d7e73ac0: Status 404 returned error can't find the container with id 12ed4119b3d4a8ef45bcd762fc72e17844c14aeb5f6362026d7aeed1d7e73ac0 Jul 29 15:14:59.235238 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:59.235201 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80fc9d-9qjc6" event={"ID":"9fed510e-03c0-4030-80ba-f3c493601374","Type":"ContainerStarted","Data":"94347ffef05968f5372e2d8bda7c9426aab5503dfaaee28f0dbbef123984060f"} Jul 29 15:14:59.235390 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:59.235255 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80fc9d-9qjc6" event={"ID":"9fed510e-03c0-4030-80ba-f3c493601374","Type":"ContainerStarted","Data":"12ed4119b3d4a8ef45bcd762fc72e17844c14aeb5f6362026d7aeed1d7e73ac0"} Jul 29 15:14:59.251995 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:14:59.251943 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80fc9d-9qjc6" podStartSLOduration=1.2519258899999999 podStartE2EDuration="1.25192589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:14:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:14:59.251396437 +0000 UTC m=+2375.979545627" watchObservedRunningTime="2026-07-29 15:14:59.25192589 +0000 UTC m=+2375.980075076" Jul 29 15:15:00.136150 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:00.136108 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755635-rkn2r"] Jul 29 15:15:00.139909 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:00.139885 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-80fc9d-29755635-lr7fp"] Jul 29 15:15:00.140062 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:00.140020 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755635-rkn2r" Jul 29 15:15:00.143171 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:00.143152 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-80fc9d-29755635-lr7fp" Jul 29 15:15:00.147094 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:00.147067 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755635-rkn2r"] Jul 29 15:15:00.154952 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:00.154928 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755635-rkn2r" Jul 29 15:15:00.164511 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:00.163392 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-80fc9d-29755635-lr7fp"] Jul 29 15:15:00.164669 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:00.164587 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-80fc9d-29755635-lr7fp" Jul 29 15:15:00.244882 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:00.244559 2580 generic.go:356] "Generic (PLEG): container finished" podID="9fed510e-03c0-4030-80ba-f3c493601374" containerID="94347ffef05968f5372e2d8bda7c9426aab5503dfaaee28f0dbbef123984060f" exitCode=0 Jul 29 15:15:00.244882 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:00.244691 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80fc9d-9qjc6" event={"ID":"9fed510e-03c0-4030-80ba-f3c493601374","Type":"ContainerDied","Data":"94347ffef05968f5372e2d8bda7c9426aab5503dfaaee28f0dbbef123984060f"} Jul 29 15:15:00.337227 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:00.337203 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-80fc9d-29755635-lr7fp"] Jul 29 15:15:00.339018 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:15:00.338991 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f2334bd_6ba5_439e_8c9c_d7a1393d100b.slice/crio-0f2699080fc1c4ddd95c78e0f858869cb2476f62fcc49915de86b45484ba6992 WatchSource:0}: Error finding container 0f2699080fc1c4ddd95c78e0f858869cb2476f62fcc49915de86b45484ba6992: Status 404 returned error can't find the container with id 0f2699080fc1c4ddd95c78e0f858869cb2476f62fcc49915de86b45484ba6992 Jul 29 15:15:00.521538 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:00.521511 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755635-rkn2r"] Jul 29 15:15:00.523479 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:15:00.523449 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93fcc4e2_d3e1_47e2_be9e_480081fda165.slice/crio-ccdc4761a0f44f6380a723ac8e25bfb081f5b0fdc57953309c9d532d10a6e6f6 WatchSource:0}: Error finding container ccdc4761a0f44f6380a723ac8e25bfb081f5b0fdc57953309c9d532d10a6e6f6: Status 404 returned error can't find the container with id ccdc4761a0f44f6380a723ac8e25bfb081f5b0fdc57953309c9d532d10a6e6f6 Jul 29 15:15:01.252479 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:01.252439 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755635-rkn2r" event={"ID":"93fcc4e2-d3e1-47e2-be9e-480081fda165","Type":"ContainerStarted","Data":"6d851edb9e757a9416ef9d5c1663b94d935abaf919caac43b9dca8eef490d7bc"} Jul 29 15:15:01.253028 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:01.252506 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755635-rkn2r" event={"ID":"93fcc4e2-d3e1-47e2-be9e-480081fda165","Type":"ContainerStarted","Data":"ccdc4761a0f44f6380a723ac8e25bfb081f5b0fdc57953309c9d532d10a6e6f6"} Jul 29 15:15:01.254152 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:01.254125 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-80fc9d-29755635-lr7fp" event={"ID":"4f2334bd-6ba5-439e-8c9c-d7a1393d100b","Type":"ContainerStarted","Data":"65bbfb99a173dd13eb7fb52790bcbae004fed166d369e041b42faed0d16d5d55"} Jul 29 15:15:01.254306 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:01.254171 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-80fc9d-29755635-lr7fp" event={"ID":"4f2334bd-6ba5-439e-8c9c-d7a1393d100b","Type":"ContainerStarted","Data":"0f2699080fc1c4ddd95c78e0f858869cb2476f62fcc49915de86b45484ba6992"} Jul 29 15:15:01.266066 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:01.266008 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755635-rkn2r" podStartSLOduration=1.265991161 podStartE2EDuration="1.265991161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:15:01.265835085 +0000 UTC m=+2377.993984264" watchObservedRunningTime="2026-07-29 15:15:01.265991161 +0000 UTC m=+2377.994140349" Jul 29 15:15:01.280324 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:01.280268 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-80fc9d-29755635-lr7fp" podStartSLOduration=1.28025054 podStartE2EDuration="1.28025054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:15:01.277849988 +0000 UTC m=+2378.005999198" watchObservedRunningTime="2026-07-29 15:15:01.28025054 +0000 UTC m=+2378.008399785" Jul 29 15:15:01.417938 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:01.417911 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80fc9d-9qjc6" Jul 29 15:15:01.489087 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:01.489056 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fed510e-03c0-4030-80ba-f3c493601374-maas-api-service-ca\") pod \"9fed510e-03c0-4030-80ba-f3c493601374\" (UID: \"9fed510e-03c0-4030-80ba-f3c493601374\") " Jul 29 15:15:01.489415 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:01.489392 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fed510e-03c0-4030-80ba-f3c493601374-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9fed510e-03c0-4030-80ba-f3c493601374" (UID: "9fed510e-03c0-4030-80ba-f3c493601374"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:15:01.590043 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:01.590002 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fed510e-03c0-4030-80ba-f3c493601374-maas-api-service-ca\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:15:02.261949 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:02.261909 2580 generic.go:356] "Generic (PLEG): container finished" podID="4f2334bd-6ba5-439e-8c9c-d7a1393d100b" containerID="65bbfb99a173dd13eb7fb52790bcbae004fed166d369e041b42faed0d16d5d55" exitCode=0 Jul 29 15:15:02.262388 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:02.261996 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-80fc9d-29755635-lr7fp" event={"ID":"4f2334bd-6ba5-439e-8c9c-d7a1393d100b","Type":"ContainerDied","Data":"65bbfb99a173dd13eb7fb52790bcbae004fed166d369e041b42faed0d16d5d55"} Jul 29 15:15:02.263590 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:02.263558 2580 generic.go:356] "Generic (PLEG): container finished" podID="93fcc4e2-d3e1-47e2-be9e-480081fda165" containerID="6d851edb9e757a9416ef9d5c1663b94d935abaf919caac43b9dca8eef490d7bc" exitCode=0 Jul 29 15:15:02.263709 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:02.263599 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755635-rkn2r" event={"ID":"93fcc4e2-d3e1-47e2-be9e-480081fda165","Type":"ContainerDied","Data":"6d851edb9e757a9416ef9d5c1663b94d935abaf919caac43b9dca8eef490d7bc"} Jul 29 15:15:02.265021 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:02.265000 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80fc9d-9qjc6" event={"ID":"9fed510e-03c0-4030-80ba-f3c493601374","Type":"ContainerDied","Data":"12ed4119b3d4a8ef45bcd762fc72e17844c14aeb5f6362026d7aeed1d7e73ac0"} Jul 29 15:15:02.265155 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:02.265009 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80fc9d-9qjc6" Jul 29 15:15:02.265155 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:02.265035 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="12ed4119b3d4a8ef45bcd762fc72e17844c14aeb5f6362026d7aeed1d7e73ac0" Jul 29 15:15:03.401784 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:03.401754 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-80fc9d-29755635-lr7fp" Jul 29 15:15:03.445886 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:03.445863 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755635-rkn2r" Jul 29 15:15:04.276635 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:04.276604 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-80fc9d-29755635-lr7fp" Jul 29 15:15:04.276851 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:04.276636 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-80fc9d-29755635-lr7fp" event={"ID":"4f2334bd-6ba5-439e-8c9c-d7a1393d100b","Type":"ContainerDied","Data":"0f2699080fc1c4ddd95c78e0f858869cb2476f62fcc49915de86b45484ba6992"} Jul 29 15:15:04.276851 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:04.276684 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f2699080fc1c4ddd95c78e0f858869cb2476f62fcc49915de86b45484ba6992" Jul 29 15:15:04.277952 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:04.277929 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755635-rkn2r" event={"ID":"93fcc4e2-d3e1-47e2-be9e-480081fda165","Type":"ContainerDied","Data":"ccdc4761a0f44f6380a723ac8e25bfb081f5b0fdc57953309c9d532d10a6e6f6"} Jul 29 15:15:04.277952 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:04.277953 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755635-rkn2r" Jul 29 15:15:04.278100 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:04.277966 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ccdc4761a0f44f6380a723ac8e25bfb081f5b0fdc57953309c9d532d10a6e6f6" Jul 29 15:15:08.810700 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:08.810660 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:08.811711 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:08.811684 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-80fc9d-29755635-lr7fp"] Jul 29 15:15:08.814208 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:08.814187 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-80fc9d-29755635-lr7fp"] Jul 29 15:15:08.850749 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:08.847976 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l"] Jul 29 15:15:08.850749 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:08.848226 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l" podUID="9e91b19c-6d99-4482-a028-c287f1a01d89" containerName="payload-processing" containerID="cri-o://4dce553475babe4904841c3560aaeed48638cd832b0bcdfc8ab817f42b7423f5" gracePeriod=30 Jul 29 15:15:08.862275 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:08.862232 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd"] Jul 29 15:15:08.862790 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:08.862698 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd" podUID="fe003141-6a4c-4351-ba88-132a5c6cd083" containerName="payload-pre-processing" containerID="cri-o://1c7bd918d5b556361e05b6c6aba13d3bcd95b445096b55644cb4cddf5bcc3057" gracePeriod=30 Jul 29 15:15:08.891940 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:08.891905 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:08.950844 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:08.950798 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:08.998201 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:08.998168 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80fc9d-9qjc6"] Jul 29 15:15:09.004602 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.004568 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80fc9d-9qjc6"] Jul 29 15:15:09.138387 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.138367 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l" Jul 29 15:15:09.141177 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.141156 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd" Jul 29 15:15:09.262671 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.262634 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e91b19c-6d99-4482-a028-c287f1a01d89-plugins-config-volume\") pod \"9e91b19c-6d99-4482-a028-c287f1a01d89\" (UID: \"9e91b19c-6d99-4482-a028-c287f1a01d89\") " Jul 29 15:15:09.262878 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.262689 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kngt4\" (UniqueName: \"kubernetes.io/projected/9e91b19c-6d99-4482-a028-c287f1a01d89-kube-api-access-kngt4\") pod \"9e91b19c-6d99-4482-a028-c287f1a01d89\" (UID: \"9e91b19c-6d99-4482-a028-c287f1a01d89\") " Jul 29 15:15:09.262878 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.262746 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe003141-6a4c-4351-ba88-132a5c6cd083-plugins-config-volume\") pod \"fe003141-6a4c-4351-ba88-132a5c6cd083\" (UID: \"fe003141-6a4c-4351-ba88-132a5c6cd083\") " Jul 29 15:15:09.262878 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.262789 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9bxk\" (UniqueName: \"kubernetes.io/projected/fe003141-6a4c-4351-ba88-132a5c6cd083-kube-api-access-j9bxk\") pod \"fe003141-6a4c-4351-ba88-132a5c6cd083\" (UID: \"fe003141-6a4c-4351-ba88-132a5c6cd083\") " Jul 29 15:15:09.263115 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.263091 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe003141-6a4c-4351-ba88-132a5c6cd083-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fe003141-6a4c-4351-ba88-132a5c6cd083" (UID: "fe003141-6a4c-4351-ba88-132a5c6cd083"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:15:09.263173 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.263113 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e91b19c-6d99-4482-a028-c287f1a01d89-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9e91b19c-6d99-4482-a028-c287f1a01d89" (UID: "9e91b19c-6d99-4482-a028-c287f1a01d89"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:15:09.264828 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.264806 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe003141-6a4c-4351-ba88-132a5c6cd083-kube-api-access-j9bxk" (OuterVolumeSpecName: "kube-api-access-j9bxk") pod "fe003141-6a4c-4351-ba88-132a5c6cd083" (UID: "fe003141-6a4c-4351-ba88-132a5c6cd083"). InnerVolumeSpecName "kube-api-access-j9bxk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:15:09.265075 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.265044 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e91b19c-6d99-4482-a028-c287f1a01d89-kube-api-access-kngt4" (OuterVolumeSpecName: "kube-api-access-kngt4") pod "9e91b19c-6d99-4482-a028-c287f1a01d89" (UID: "9e91b19c-6d99-4482-a028-c287f1a01d89"). InnerVolumeSpecName "kube-api-access-kngt4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:15:09.309145 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.309112 2580 generic.go:356] "Generic (PLEG): container finished" podID="fe003141-6a4c-4351-ba88-132a5c6cd083" containerID="1c7bd918d5b556361e05b6c6aba13d3bcd95b445096b55644cb4cddf5bcc3057" exitCode=0 Jul 29 15:15:09.309329 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.309159 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd" Jul 29 15:15:09.309329 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.309203 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd" event={"ID":"fe003141-6a4c-4351-ba88-132a5c6cd083","Type":"ContainerDied","Data":"1c7bd918d5b556361e05b6c6aba13d3bcd95b445096b55644cb4cddf5bcc3057"} Jul 29 15:15:09.309329 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.309260 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd" event={"ID":"fe003141-6a4c-4351-ba88-132a5c6cd083","Type":"ContainerDied","Data":"e35d6ee5c7039609bb8b9b65e32f8c10b02364a133b99b3838b1791db50f1320"} Jul 29 15:15:09.309329 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.309290 2580 scope.go:117] "RemoveContainer" containerID="1c7bd918d5b556361e05b6c6aba13d3bcd95b445096b55644cb4cddf5bcc3057" Jul 29 15:15:09.310585 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.310561 2580 generic.go:356] "Generic (PLEG): container finished" podID="9e91b19c-6d99-4482-a028-c287f1a01d89" containerID="4dce553475babe4904841c3560aaeed48638cd832b0bcdfc8ab817f42b7423f5" exitCode=0 Jul 29 15:15:09.310706 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.310606 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l" event={"ID":"9e91b19c-6d99-4482-a028-c287f1a01d89","Type":"ContainerDied","Data":"4dce553475babe4904841c3560aaeed48638cd832b0bcdfc8ab817f42b7423f5"} Jul 29 15:15:09.310706 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.310625 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l" Jul 29 15:15:09.310706 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.310644 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l" event={"ID":"9e91b19c-6d99-4482-a028-c287f1a01d89","Type":"ContainerDied","Data":"08288c88cef8dd897d002485b96a233dddf50cf6b3133fe60d215d3572344ead"} Jul 29 15:15:09.319849 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.319822 2580 scope.go:117] "RemoveContainer" containerID="1c7bd918d5b556361e05b6c6aba13d3bcd95b445096b55644cb4cddf5bcc3057" Jul 29 15:15:09.320132 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:15:09.320114 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c7bd918d5b556361e05b6c6aba13d3bcd95b445096b55644cb4cddf5bcc3057\": container with ID starting with 1c7bd918d5b556361e05b6c6aba13d3bcd95b445096b55644cb4cddf5bcc3057 not found: ID does not exist" containerID="1c7bd918d5b556361e05b6c6aba13d3bcd95b445096b55644cb4cddf5bcc3057" Jul 29 15:15:09.320220 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.320140 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c7bd918d5b556361e05b6c6aba13d3bcd95b445096b55644cb4cddf5bcc3057"} err="failed to get container status \"1c7bd918d5b556361e05b6c6aba13d3bcd95b445096b55644cb4cddf5bcc3057\": rpc error: code = NotFound desc = could not find container \"1c7bd918d5b556361e05b6c6aba13d3bcd95b445096b55644cb4cddf5bcc3057\": container with ID starting with 1c7bd918d5b556361e05b6c6aba13d3bcd95b445096b55644cb4cddf5bcc3057 not found: ID does not exist" Jul 29 15:15:09.320220 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.320156 2580 scope.go:117] "RemoveContainer" containerID="4dce553475babe4904841c3560aaeed48638cd832b0bcdfc8ab817f42b7423f5" Jul 29 15:15:09.331321 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.331237 2580 scope.go:117] "RemoveContainer" containerID="4dce553475babe4904841c3560aaeed48638cd832b0bcdfc8ab817f42b7423f5" Jul 29 15:15:09.331687 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:15:09.331628 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dce553475babe4904841c3560aaeed48638cd832b0bcdfc8ab817f42b7423f5\": container with ID starting with 4dce553475babe4904841c3560aaeed48638cd832b0bcdfc8ab817f42b7423f5 not found: ID does not exist" containerID="4dce553475babe4904841c3560aaeed48638cd832b0bcdfc8ab817f42b7423f5" Jul 29 15:15:09.331687 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.331663 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dce553475babe4904841c3560aaeed48638cd832b0bcdfc8ab817f42b7423f5"} err="failed to get container status \"4dce553475babe4904841c3560aaeed48638cd832b0bcdfc8ab817f42b7423f5\": rpc error: code = NotFound desc = could not find container \"4dce553475babe4904841c3560aaeed48638cd832b0bcdfc8ab817f42b7423f5\": container with ID starting with 4dce553475babe4904841c3560aaeed48638cd832b0bcdfc8ab817f42b7423f5 not found: ID does not exist" Jul 29 15:15:09.333898 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.333879 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd"] Jul 29 15:15:09.337870 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.337850 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-80fc9d-79c465c864-gp9sd"] Jul 29 15:15:09.350055 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.350024 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l"] Jul 29 15:15:09.360139 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.360115 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-80fc9d-7d9bd6b4cb-bhj7l"] Jul 29 15:15:09.363522 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.363501 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe003141-6a4c-4351-ba88-132a5c6cd083-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:15:09.363522 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.363523 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9bxk\" (UniqueName: \"kubernetes.io/projected/fe003141-6a4c-4351-ba88-132a5c6cd083-kube-api-access-j9bxk\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:15:09.363677 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.363535 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e91b19c-6d99-4482-a028-c287f1a01d89-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:15:09.363677 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.363546 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kngt4\" (UniqueName: \"kubernetes.io/projected/9e91b19c-6d99-4482-a028-c287f1a01d89-kube-api-access-kngt4\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:15:09.817783 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.817745 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f2334bd-6ba5-439e-8c9c-d7a1393d100b" path="/var/lib/kubelet/pods/4f2334bd-6ba5-439e-8c9c-d7a1393d100b/volumes" Jul 29 15:15:09.818188 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.818119 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e91b19c-6d99-4482-a028-c287f1a01d89" path="/var/lib/kubelet/pods/9e91b19c-6d99-4482-a028-c287f1a01d89/volumes" Jul 29 15:15:09.818416 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.818403 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fed510e-03c0-4030-80ba-f3c493601374" path="/var/lib/kubelet/pods/9fed510e-03c0-4030-80ba-f3c493601374/volumes" Jul 29 15:15:09.818714 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:09.818703 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe003141-6a4c-4351-ba88-132a5c6cd083" path="/var/lib/kubelet/pods/fe003141-6a4c-4351-ba88-132a5c6cd083/volumes" Jul 29 15:15:10.589824 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:10.589778 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:10.699612 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:10.699579 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:15.239298 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:15.239262 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:15.304882 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:15.304842 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:15.517523 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:15.517481 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:15.745084 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:15.745044 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:16.100985 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:16.100953 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:16.416362 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:16.416269 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:16.657828 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:16.657796 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:19.814559 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:19.814524 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:19.911834 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:19.911801 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:20.073496 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.073405 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:20.463198 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.463083 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:20.579200 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.579168 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:20.680712 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.680668 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:20.695089 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.695059 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff"] Jul 29 15:15:20.695446 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.695434 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93fcc4e2-d3e1-47e2-be9e-480081fda165" containerName="cleanup" Jul 29 15:15:20.695506 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.695449 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="93fcc4e2-d3e1-47e2-be9e-480081fda165" containerName="cleanup" Jul 29 15:15:20.695506 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.695462 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e91b19c-6d99-4482-a028-c287f1a01d89" containerName="payload-processing" Jul 29 15:15:20.695506 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.695467 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e91b19c-6d99-4482-a028-c287f1a01d89" containerName="payload-processing" Jul 29 15:15:20.695506 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.695482 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f2334bd-6ba5-439e-8c9c-d7a1393d100b" containerName="cleanup" Jul 29 15:15:20.695506 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.695488 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f2334bd-6ba5-439e-8c9c-d7a1393d100b" containerName="cleanup" Jul 29 15:15:20.695506 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.695496 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe003141-6a4c-4351-ba88-132a5c6cd083" containerName="payload-pre-processing" Jul 29 15:15:20.695506 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.695501 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe003141-6a4c-4351-ba88-132a5c6cd083" containerName="payload-pre-processing" Jul 29 15:15:20.695506 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.695507 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fed510e-03c0-4030-80ba-f3c493601374" containerName="revoke-keys" Jul 29 15:15:20.695769 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.695512 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fed510e-03c0-4030-80ba-f3c493601374" containerName="revoke-keys" Jul 29 15:15:20.695769 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.695566 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e91b19c-6d99-4482-a028-c287f1a01d89" containerName="payload-processing" Jul 29 15:15:20.695769 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.695575 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe003141-6a4c-4351-ba88-132a5c6cd083" containerName="payload-pre-processing" Jul 29 15:15:20.695769 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.695583 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="93fcc4e2-d3e1-47e2-be9e-480081fda165" containerName="cleanup" Jul 29 15:15:20.695769 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.695590 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fed510e-03c0-4030-80ba-f3c493601374" containerName="revoke-keys" Jul 29 15:15:20.695769 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.695595 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f2334bd-6ba5-439e-8c9c-d7a1393d100b" containerName="cleanup" Jul 29 15:15:20.700218 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.700195 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff" Jul 29 15:15:20.702497 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.702470 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-baf2cc\"" Jul 29 15:15:20.713761 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.713656 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff"] Jul 29 15:15:20.726830 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.726795 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv"] Jul 29 15:15:20.730600 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.730577 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv" Jul 29 15:15:20.733141 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.733112 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-baf2cc-dockercfg-ks96j\"" Jul 29 15:15:20.742347 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.742322 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv"] Jul 29 15:15:20.771741 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.771695 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4nv4\" (UniqueName: \"kubernetes.io/projected/d38cebf3-bf9a-482a-88f1-73638880723b-kube-api-access-w4nv4\") pod \"payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv\" (UID: \"d38cebf3-bf9a-482a-88f1-73638880723b\") " pod="openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv" Jul 29 15:15:20.771973 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.771767 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d38cebf3-bf9a-482a-88f1-73638880723b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv\" (UID: \"d38cebf3-bf9a-482a-88f1-73638880723b\") " pod="openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv" Jul 29 15:15:20.771973 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.771847 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4csr\" (UniqueName: \"kubernetes.io/projected/b995fd88-9940-447f-b7b9-c2ae59167eaa-kube-api-access-m4csr\") pod \"payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff\" (UID: \"b995fd88-9940-447f-b7b9-c2ae59167eaa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff" Jul 29 15:15:20.771973 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.771939 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b995fd88-9940-447f-b7b9-c2ae59167eaa-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff\" (UID: \"b995fd88-9940-447f-b7b9-c2ae59167eaa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff" Jul 29 15:15:20.873005 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.872964 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b995fd88-9940-447f-b7b9-c2ae59167eaa-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff\" (UID: \"b995fd88-9940-447f-b7b9-c2ae59167eaa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff" Jul 29 15:15:20.873452 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.873107 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4nv4\" (UniqueName: \"kubernetes.io/projected/d38cebf3-bf9a-482a-88f1-73638880723b-kube-api-access-w4nv4\") pod \"payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv\" (UID: \"d38cebf3-bf9a-482a-88f1-73638880723b\") " pod="openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv" Jul 29 15:15:20.873452 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.873130 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d38cebf3-bf9a-482a-88f1-73638880723b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv\" (UID: \"d38cebf3-bf9a-482a-88f1-73638880723b\") " pod="openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv" Jul 29 15:15:20.873452 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.873160 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4csr\" (UniqueName: \"kubernetes.io/projected/b995fd88-9940-447f-b7b9-c2ae59167eaa-kube-api-access-m4csr\") pod \"payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff\" (UID: \"b995fd88-9940-447f-b7b9-c2ae59167eaa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff" Jul 29 15:15:20.873849 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.873832 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d38cebf3-bf9a-482a-88f1-73638880723b-plugins-config-volume\") pod \"payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv\" (UID: \"d38cebf3-bf9a-482a-88f1-73638880723b\") " pod="openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv" Jul 29 15:15:20.873849 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.873841 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b995fd88-9940-447f-b7b9-c2ae59167eaa-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff\" (UID: \"b995fd88-9940-447f-b7b9-c2ae59167eaa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff" Jul 29 15:15:20.885222 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.885185 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4csr\" (UniqueName: \"kubernetes.io/projected/b995fd88-9940-447f-b7b9-c2ae59167eaa-kube-api-access-m4csr\") pod \"payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff\" (UID: \"b995fd88-9940-447f-b7b9-c2ae59167eaa\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff" Jul 29 15:15:20.888018 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:20.887990 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4nv4\" (UniqueName: \"kubernetes.io/projected/d38cebf3-bf9a-482a-88f1-73638880723b-kube-api-access-w4nv4\") pod \"payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv\" (UID: \"d38cebf3-bf9a-482a-88f1-73638880723b\") " pod="openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv" Jul 29 15:15:21.014920 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:21.014882 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff" Jul 29 15:15:21.041901 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:21.041869 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv" Jul 29 15:15:21.177171 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:21.176766 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff"] Jul 29 15:15:21.180109 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:15:21.180078 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb995fd88_9940_447f_b7b9_c2ae59167eaa.slice/crio-024446b8ef7cc70ff2cc285980cb2d71d86ea6e94b31ebfd900352edef16e611 WatchSource:0}: Error finding container 024446b8ef7cc70ff2cc285980cb2d71d86ea6e94b31ebfd900352edef16e611: Status 404 returned error can't find the container with id 024446b8ef7cc70ff2cc285980cb2d71d86ea6e94b31ebfd900352edef16e611 Jul 29 15:15:21.205820 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:21.205788 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv"] Jul 29 15:15:21.207281 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:15:21.207254 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd38cebf3_bf9a_482a_88f1_73638880723b.slice/crio-2c13df15fc6c61d6437591695a06079c552ed925855ad9f6b5c79def44a67465 WatchSource:0}: Error finding container 2c13df15fc6c61d6437591695a06079c552ed925855ad9f6b5c79def44a67465: Status 404 returned error can't find the container with id 2c13df15fc6c61d6437591695a06079c552ed925855ad9f6b5c79def44a67465 Jul 29 15:15:21.387038 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:21.386930 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv" event={"ID":"d38cebf3-bf9a-482a-88f1-73638880723b","Type":"ContainerStarted","Data":"248ef95bb48b4da2250d8ce49ba7d5a9488bc0daff9fa9f0deae063f14fa59fe"} Jul 29 15:15:21.387038 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:21.386988 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv" event={"ID":"d38cebf3-bf9a-482a-88f1-73638880723b","Type":"ContainerStarted","Data":"2c13df15fc6c61d6437591695a06079c552ed925855ad9f6b5c79def44a67465"} Jul 29 15:15:21.387038 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:21.387010 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv" Jul 29 15:15:21.388529 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:21.388505 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff" event={"ID":"b995fd88-9940-447f-b7b9-c2ae59167eaa","Type":"ContainerStarted","Data":"4e70d618350085c5861e852045f9de20bd5941cf54faf1a672f188852d9427a3"} Jul 29 15:15:21.388647 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:21.388543 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff" event={"ID":"b995fd88-9940-447f-b7b9-c2ae59167eaa","Type":"ContainerStarted","Data":"024446b8ef7cc70ff2cc285980cb2d71d86ea6e94b31ebfd900352edef16e611"} Jul 29 15:15:21.388647 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:21.388620 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff" Jul 29 15:15:21.403382 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:21.403335 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv" podStartSLOduration=1.403319557 podStartE2EDuration="1.403319557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:15:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:15:21.401866547 +0000 UTC m=+2398.130015748" watchObservedRunningTime="2026-07-29 15:15:21.403319557 +0000 UTC m=+2398.131468746" Jul 29 15:15:21.419266 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:21.419203 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff" podStartSLOduration=1.419185534 podStartE2EDuration="1.419185534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:15:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:15:21.416479468 +0000 UTC m=+2398.144628649" watchObservedRunningTime="2026-07-29 15:15:21.419185534 +0000 UTC m=+2398.147334748" Jul 29 15:15:23.927401 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:23.927371 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 15:15:23.933179 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:23.933158 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 15:15:32.399491 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:32.399459 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff" Jul 29 15:15:32.400023 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:32.399651 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv" Jul 29 15:15:44.848261 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:44.848211 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:45.330447 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:45.330358 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:45.688906 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:45.688801 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:47.738029 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:47.737988 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:48.961543 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:48.961506 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:49.044018 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:49.043971 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:49.116101 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:49.116066 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:49.440644 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:49.440597 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:49.645185 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:49.645151 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:49.746303 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:49.746257 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:15:49.787715 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:49.787673 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww"] Jul 29 15:15:49.791697 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:49.791674 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww" Jul 29 15:15:49.793804 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:49.793712 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-d3e823\"" Jul 29 15:15:49.800623 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:49.800602 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww"] Jul 29 15:15:49.803292 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:49.803272 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv"] Jul 29 15:15:49.806754 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:49.806736 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv" Jul 29 15:15:49.808972 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:49.808946 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-d3e823-dockercfg-d76v5\"" Jul 29 15:15:49.821764 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:49.821744 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv"] Jul 29 15:15:49.931888 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:49.931848 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b8lf\" (UniqueName: \"kubernetes.io/projected/11b8fc23-d44f-4660-91ad-a9036a935d68-kube-api-access-6b8lf\") pod \"payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww\" (UID: \"11b8fc23-d44f-4660-91ad-a9036a935d68\") " pod="openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww" Jul 29 15:15:49.932077 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:49.931899 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhmc9\" (UniqueName: \"kubernetes.io/projected/5403194b-dbe8-4e8b-961b-b71cc153f767-kube-api-access-vhmc9\") pod \"payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv\" (UID: \"5403194b-dbe8-4e8b-961b-b71cc153f767\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv" Jul 29 15:15:49.932077 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:49.931954 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5403194b-dbe8-4e8b-961b-b71cc153f767-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv\" (UID: \"5403194b-dbe8-4e8b-961b-b71cc153f767\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv" Jul 29 15:15:49.932351 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:49.932311 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11b8fc23-d44f-4660-91ad-a9036a935d68-plugins-config-volume\") pod \"payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww\" (UID: \"11b8fc23-d44f-4660-91ad-a9036a935d68\") " pod="openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww" Jul 29 15:15:50.033165 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:50.033070 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11b8fc23-d44f-4660-91ad-a9036a935d68-plugins-config-volume\") pod \"payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww\" (UID: \"11b8fc23-d44f-4660-91ad-a9036a935d68\") " pod="openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww" Jul 29 15:15:50.033165 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:50.033119 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b8lf\" (UniqueName: \"kubernetes.io/projected/11b8fc23-d44f-4660-91ad-a9036a935d68-kube-api-access-6b8lf\") pod \"payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww\" (UID: \"11b8fc23-d44f-4660-91ad-a9036a935d68\") " pod="openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww" Jul 29 15:15:50.033165 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:50.033151 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhmc9\" (UniqueName: \"kubernetes.io/projected/5403194b-dbe8-4e8b-961b-b71cc153f767-kube-api-access-vhmc9\") pod \"payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv\" (UID: \"5403194b-dbe8-4e8b-961b-b71cc153f767\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv" Jul 29 15:15:50.034270 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:50.033183 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5403194b-dbe8-4e8b-961b-b71cc153f767-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv\" (UID: \"5403194b-dbe8-4e8b-961b-b71cc153f767\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv" Jul 29 15:15:50.034270 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:50.034119 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11b8fc23-d44f-4660-91ad-a9036a935d68-plugins-config-volume\") pod \"payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww\" (UID: \"11b8fc23-d44f-4660-91ad-a9036a935d68\") " pod="openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww" Jul 29 15:15:50.034270 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:50.034220 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5403194b-dbe8-4e8b-961b-b71cc153f767-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv\" (UID: \"5403194b-dbe8-4e8b-961b-b71cc153f767\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv" Jul 29 15:15:50.042017 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:50.041992 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhmc9\" (UniqueName: \"kubernetes.io/projected/5403194b-dbe8-4e8b-961b-b71cc153f767-kube-api-access-vhmc9\") pod \"payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv\" (UID: \"5403194b-dbe8-4e8b-961b-b71cc153f767\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv" Jul 29 15:15:50.042137 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:50.042024 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b8lf\" (UniqueName: \"kubernetes.io/projected/11b8fc23-d44f-4660-91ad-a9036a935d68-kube-api-access-6b8lf\") pod \"payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww\" (UID: \"11b8fc23-d44f-4660-91ad-a9036a935d68\") " pod="openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww" Jul 29 15:15:50.104026 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:50.103988 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww" Jul 29 15:15:50.118486 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:50.118455 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv" Jul 29 15:15:50.262024 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:50.261951 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww"] Jul 29 15:15:50.263963 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:15:50.263922 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11b8fc23_d44f_4660_91ad_a9036a935d68.slice/crio-09fdb7340180ce20010988258aa67a72260cb2c4e5f6fce6a3fb0b415b74a18c WatchSource:0}: Error finding container 09fdb7340180ce20010988258aa67a72260cb2c4e5f6fce6a3fb0b415b74a18c: Status 404 returned error can't find the container with id 09fdb7340180ce20010988258aa67a72260cb2c4e5f6fce6a3fb0b415b74a18c Jul 29 15:15:50.292011 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:50.291982 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv"] Jul 29 15:15:50.302789 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:15:50.302761 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5403194b_dbe8_4e8b_961b_b71cc153f767.slice/crio-e34628663506c4fefddf92133c1a2947481a6fb6881c255fcc28f8b6bf758830 WatchSource:0}: Error finding container e34628663506c4fefddf92133c1a2947481a6fb6881c255fcc28f8b6bf758830: Status 404 returned error can't find the container with id e34628663506c4fefddf92133c1a2947481a6fb6881c255fcc28f8b6bf758830 Jul 29 15:15:50.555880 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:50.555764 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww" event={"ID":"11b8fc23-d44f-4660-91ad-a9036a935d68","Type":"ContainerStarted","Data":"4e16fa4d11683230cbaf5e68b0bfab7f3e376736f709c142987f243b50ea7e9e"} Jul 29 15:15:50.555880 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:50.555853 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww" Jul 29 15:15:50.555880 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:50.555878 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww" event={"ID":"11b8fc23-d44f-4660-91ad-a9036a935d68","Type":"ContainerStarted","Data":"09fdb7340180ce20010988258aa67a72260cb2c4e5f6fce6a3fb0b415b74a18c"} Jul 29 15:15:50.557715 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:50.557686 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv" event={"ID":"5403194b-dbe8-4e8b-961b-b71cc153f767","Type":"ContainerStarted","Data":"5089d4b2e82ef6f979db129c849de753d2a4747877b82b7de7c2c57daed41965"} Jul 29 15:15:50.557851 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:50.557777 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv" Jul 29 15:15:50.557851 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:50.557802 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv" event={"ID":"5403194b-dbe8-4e8b-961b-b71cc153f767","Type":"ContainerStarted","Data":"e34628663506c4fefddf92133c1a2947481a6fb6881c255fcc28f8b6bf758830"} Jul 29 15:15:50.572509 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:50.572460 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww" podStartSLOduration=1.572442445 podStartE2EDuration="1.572442445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:15:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:15:50.570370003 +0000 UTC m=+2427.298519229" watchObservedRunningTime="2026-07-29 15:15:50.572442445 +0000 UTC m=+2427.300591636" Jul 29 15:15:50.587497 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:15:50.587444 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv" podStartSLOduration=1.587430307 podStartE2EDuration="1.587430307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:15:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:15:50.584416885 +0000 UTC m=+2427.312566074" watchObservedRunningTime="2026-07-29 15:15:50.587430307 +0000 UTC m=+2427.315579532" Jul 29 15:16:01.566916 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:01.566883 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww" Jul 29 15:16:01.567364 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:01.567191 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv" Jul 29 15:16:13.982626 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:13.982579 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:16:17.248706 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:17.248669 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-baf2cc-zg79l"] Jul 29 15:16:17.252284 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:17.252264 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-baf2cc-zg79l" Jul 29 15:16:17.254278 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:17.254259 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mmt6p\"" Jul 29 15:16:17.254387 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:17.254300 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 29 15:16:17.259569 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:17.259531 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-baf2cc-zg79l"] Jul 29 15:16:17.290859 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:17.290818 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00af3b81-cdde-4567-9e57-94bc92d5b90f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-baf2cc-zg79l\" (UID: \"00af3b81-cdde-4567-9e57-94bc92d5b90f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-baf2cc-zg79l" Jul 29 15:16:17.391966 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:17.391926 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00af3b81-cdde-4567-9e57-94bc92d5b90f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-baf2cc-zg79l\" (UID: \"00af3b81-cdde-4567-9e57-94bc92d5b90f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-baf2cc-zg79l" Jul 29 15:16:17.392594 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:17.392568 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00af3b81-cdde-4567-9e57-94bc92d5b90f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-baf2cc-zg79l\" (UID: \"00af3b81-cdde-4567-9e57-94bc92d5b90f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-baf2cc-zg79l" Jul 29 15:16:17.565435 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:17.565343 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-baf2cc-zg79l" Jul 29 15:16:17.712235 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:17.712192 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-baf2cc-zg79l"] Jul 29 15:16:17.719656 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:16:17.719628 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00af3b81_cdde_4567_9e57_94bc92d5b90f.slice/crio-0b5658b63493a784909cc2b4b0e870caebe6882962040b4b7b8dea05299eeb21 WatchSource:0}: Error finding container 0b5658b63493a784909cc2b4b0e870caebe6882962040b4b7b8dea05299eeb21: Status 404 returned error can't find the container with id 0b5658b63493a784909cc2b4b0e870caebe6882962040b4b7b8dea05299eeb21 Jul 29 15:16:18.728311 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:18.728218 2580 generic.go:356] "Generic (PLEG): container finished" podID="00af3b81-cdde-4567-9e57-94bc92d5b90f" containerID="bad7c29648a2ba606f77da22d3bfab4a59317e520fa728f3e3dcba9c47b77f51" exitCode=0 Jul 29 15:16:18.728673 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:18.728306 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-baf2cc-zg79l" event={"ID":"00af3b81-cdde-4567-9e57-94bc92d5b90f","Type":"ContainerDied","Data":"bad7c29648a2ba606f77da22d3bfab4a59317e520fa728f3e3dcba9c47b77f51"} Jul 29 15:16:18.728673 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:18.728363 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-baf2cc-zg79l" event={"ID":"00af3b81-cdde-4567-9e57-94bc92d5b90f","Type":"ContainerStarted","Data":"0b5658b63493a784909cc2b4b0e870caebe6882962040b4b7b8dea05299eeb21"} Jul 29 15:16:19.873932 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:19.873905 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-baf2cc-zg79l" Jul 29 15:16:19.916030 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:19.915994 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00af3b81-cdde-4567-9e57-94bc92d5b90f-maas-api-service-ca\") pod \"00af3b81-cdde-4567-9e57-94bc92d5b90f\" (UID: \"00af3b81-cdde-4567-9e57-94bc92d5b90f\") " Jul 29 15:16:19.916384 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:19.916361 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00af3b81-cdde-4567-9e57-94bc92d5b90f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "00af3b81-cdde-4567-9e57-94bc92d5b90f" (UID: "00af3b81-cdde-4567-9e57-94bc92d5b90f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:16:20.017313 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:20.017277 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/00af3b81-cdde-4567-9e57-94bc92d5b90f-maas-api-service-ca\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:16:20.741120 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:20.741088 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-baf2cc-zg79l" Jul 29 15:16:20.741311 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:20.741117 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-baf2cc-zg79l" event={"ID":"00af3b81-cdde-4567-9e57-94bc92d5b90f","Type":"ContainerDied","Data":"0b5658b63493a784909cc2b4b0e870caebe6882962040b4b7b8dea05299eeb21"} Jul 29 15:16:20.741311 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:20.741172 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b5658b63493a784909cc2b4b0e870caebe6882962040b4b7b8dea05299eeb21" Jul 29 15:16:32.726322 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:32.726284 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:16:32.762804 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:32.762766 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv"] Jul 29 15:16:32.763062 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:32.763032 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv" podUID="d38cebf3-bf9a-482a-88f1-73638880723b" containerName="payload-processing" containerID="cri-o://248ef95bb48b4da2250d8ce49ba7d5a9488bc0daff9fa9f0deae063f14fa59fe" gracePeriod=30 Jul 29 15:16:32.766690 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:32.766636 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff"] Jul 29 15:16:32.766925 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:32.766876 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff" podUID="b995fd88-9940-447f-b7b9-c2ae59167eaa" containerName="payload-pre-processing" containerID="cri-o://4e70d618350085c5861e852045f9de20bd5941cf54faf1a672f188852d9427a3" gracePeriod=30 Jul 29 15:16:32.805984 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:32.805952 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:16:32.912968 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:32.912931 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:16:32.913146 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:32.913022 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-baf2cc-zg79l"] Jul 29 15:16:32.915498 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:32.915475 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-baf2cc-zg79l"] Jul 29 15:16:32.969910 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:32.969882 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:16:33.029308 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.029276 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv" Jul 29 15:16:33.063136 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.063112 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff" Jul 29 15:16:33.145775 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.145741 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4nv4\" (UniqueName: \"kubernetes.io/projected/d38cebf3-bf9a-482a-88f1-73638880723b-kube-api-access-w4nv4\") pod \"d38cebf3-bf9a-482a-88f1-73638880723b\" (UID: \"d38cebf3-bf9a-482a-88f1-73638880723b\") " Jul 29 15:16:33.145965 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.145802 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d38cebf3-bf9a-482a-88f1-73638880723b-plugins-config-volume\") pod \"d38cebf3-bf9a-482a-88f1-73638880723b\" (UID: \"d38cebf3-bf9a-482a-88f1-73638880723b\") " Jul 29 15:16:33.145965 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.145851 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b995fd88-9940-447f-b7b9-c2ae59167eaa-plugins-config-volume\") pod \"b995fd88-9940-447f-b7b9-c2ae59167eaa\" (UID: \"b995fd88-9940-447f-b7b9-c2ae59167eaa\") " Jul 29 15:16:33.145965 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.145877 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4csr\" (UniqueName: \"kubernetes.io/projected/b995fd88-9940-447f-b7b9-c2ae59167eaa-kube-api-access-m4csr\") pod \"b995fd88-9940-447f-b7b9-c2ae59167eaa\" (UID: \"b995fd88-9940-447f-b7b9-c2ae59167eaa\") " Jul 29 15:16:33.146200 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.146177 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d38cebf3-bf9a-482a-88f1-73638880723b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d38cebf3-bf9a-482a-88f1-73638880723b" (UID: "d38cebf3-bf9a-482a-88f1-73638880723b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:16:33.146261 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.146238 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b995fd88-9940-447f-b7b9-c2ae59167eaa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b995fd88-9940-447f-b7b9-c2ae59167eaa" (UID: "b995fd88-9940-447f-b7b9-c2ae59167eaa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:16:33.148269 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.148231 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d38cebf3-bf9a-482a-88f1-73638880723b-kube-api-access-w4nv4" (OuterVolumeSpecName: "kube-api-access-w4nv4") pod "d38cebf3-bf9a-482a-88f1-73638880723b" (UID: "d38cebf3-bf9a-482a-88f1-73638880723b"). InnerVolumeSpecName "kube-api-access-w4nv4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:16:33.148406 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.148282 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b995fd88-9940-447f-b7b9-c2ae59167eaa-kube-api-access-m4csr" (OuterVolumeSpecName: "kube-api-access-m4csr") pod "b995fd88-9940-447f-b7b9-c2ae59167eaa" (UID: "b995fd88-9940-447f-b7b9-c2ae59167eaa"). InnerVolumeSpecName "kube-api-access-m4csr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:16:33.247187 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.247150 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b995fd88-9940-447f-b7b9-c2ae59167eaa-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:16:33.247187 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.247182 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4csr\" (UniqueName: \"kubernetes.io/projected/b995fd88-9940-447f-b7b9-c2ae59167eaa-kube-api-access-m4csr\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:16:33.247389 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.247196 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4nv4\" (UniqueName: \"kubernetes.io/projected/d38cebf3-bf9a-482a-88f1-73638880723b-kube-api-access-w4nv4\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:16:33.247389 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.247208 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d38cebf3-bf9a-482a-88f1-73638880723b-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:16:33.319634 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.319597 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:16:33.818269 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.818236 2580 generic.go:356] "Generic (PLEG): container finished" podID="d38cebf3-bf9a-482a-88f1-73638880723b" containerID="248ef95bb48b4da2250d8ce49ba7d5a9488bc0daff9fa9f0deae063f14fa59fe" exitCode=0 Jul 29 15:16:33.818773 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.818350 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv" Jul 29 15:16:33.818989 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.818965 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00af3b81-cdde-4567-9e57-94bc92d5b90f" path="/var/lib/kubelet/pods/00af3b81-cdde-4567-9e57-94bc92d5b90f/volumes" Jul 29 15:16:33.819618 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.819592 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv" event={"ID":"d38cebf3-bf9a-482a-88f1-73638880723b","Type":"ContainerDied","Data":"248ef95bb48b4da2250d8ce49ba7d5a9488bc0daff9fa9f0deae063f14fa59fe"} Jul 29 15:16:33.819753 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.819637 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv" event={"ID":"d38cebf3-bf9a-482a-88f1-73638880723b","Type":"ContainerDied","Data":"2c13df15fc6c61d6437591695a06079c552ed925855ad9f6b5c79def44a67465"} Jul 29 15:16:33.819753 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.819658 2580 scope.go:117] "RemoveContainer" containerID="248ef95bb48b4da2250d8ce49ba7d5a9488bc0daff9fa9f0deae063f14fa59fe" Jul 29 15:16:33.819910 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.819878 2580 generic.go:356] "Generic (PLEG): container finished" podID="b995fd88-9940-447f-b7b9-c2ae59167eaa" containerID="4e70d618350085c5861e852045f9de20bd5941cf54faf1a672f188852d9427a3" exitCode=0 Jul 29 15:16:33.819974 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.819918 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff" event={"ID":"b995fd88-9940-447f-b7b9-c2ae59167eaa","Type":"ContainerDied","Data":"4e70d618350085c5861e852045f9de20bd5941cf54faf1a672f188852d9427a3"} Jul 29 15:16:33.820029 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.819993 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff" event={"ID":"b995fd88-9940-447f-b7b9-c2ae59167eaa","Type":"ContainerDied","Data":"024446b8ef7cc70ff2cc285980cb2d71d86ea6e94b31ebfd900352edef16e611"} Jul 29 15:16:33.820099 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.820085 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff" Jul 29 15:16:33.831751 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.831681 2580 scope.go:117] "RemoveContainer" containerID="248ef95bb48b4da2250d8ce49ba7d5a9488bc0daff9fa9f0deae063f14fa59fe" Jul 29 15:16:33.832104 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:16:33.832077 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"248ef95bb48b4da2250d8ce49ba7d5a9488bc0daff9fa9f0deae063f14fa59fe\": container with ID starting with 248ef95bb48b4da2250d8ce49ba7d5a9488bc0daff9fa9f0deae063f14fa59fe not found: ID does not exist" containerID="248ef95bb48b4da2250d8ce49ba7d5a9488bc0daff9fa9f0deae063f14fa59fe" Jul 29 15:16:33.832174 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.832116 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"248ef95bb48b4da2250d8ce49ba7d5a9488bc0daff9fa9f0deae063f14fa59fe"} err="failed to get container status \"248ef95bb48b4da2250d8ce49ba7d5a9488bc0daff9fa9f0deae063f14fa59fe\": rpc error: code = NotFound desc = could not find container \"248ef95bb48b4da2250d8ce49ba7d5a9488bc0daff9fa9f0deae063f14fa59fe\": container with ID starting with 248ef95bb48b4da2250d8ce49ba7d5a9488bc0daff9fa9f0deae063f14fa59fe not found: ID does not exist" Jul 29 15:16:33.832174 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.832142 2580 scope.go:117] "RemoveContainer" containerID="4e70d618350085c5861e852045f9de20bd5941cf54faf1a672f188852d9427a3" Jul 29 15:16:33.841072 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.841052 2580 scope.go:117] "RemoveContainer" containerID="4e70d618350085c5861e852045f9de20bd5941cf54faf1a672f188852d9427a3" Jul 29 15:16:33.841353 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:16:33.841337 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e70d618350085c5861e852045f9de20bd5941cf54faf1a672f188852d9427a3\": container with ID starting with 4e70d618350085c5861e852045f9de20bd5941cf54faf1a672f188852d9427a3 not found: ID does not exist" containerID="4e70d618350085c5861e852045f9de20bd5941cf54faf1a672f188852d9427a3" Jul 29 15:16:33.841404 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.841360 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e70d618350085c5861e852045f9de20bd5941cf54faf1a672f188852d9427a3"} err="failed to get container status \"4e70d618350085c5861e852045f9de20bd5941cf54faf1a672f188852d9427a3\": rpc error: code = NotFound desc = could not find container \"4e70d618350085c5861e852045f9de20bd5941cf54faf1a672f188852d9427a3\": container with ID starting with 4e70d618350085c5861e852045f9de20bd5941cf54faf1a672f188852d9427a3 not found: ID does not exist" Jul 29 15:16:33.853997 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.853957 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv"] Jul 29 15:16:33.858901 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.858871 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-baf2cc-6d4d84679d-khlxv"] Jul 29 15:16:33.868834 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.868805 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff"] Jul 29 15:16:33.875212 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:33.875187 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-baf2cc-fd944f988-t8cff"] Jul 29 15:16:35.817767 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:35.817711 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b995fd88-9940-447f-b7b9-c2ae59167eaa" path="/var/lib/kubelet/pods/b995fd88-9940-447f-b7b9-c2ae59167eaa/volumes" Jul 29 15:16:35.818161 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:35.818147 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d38cebf3-bf9a-482a-88f1-73638880723b" path="/var/lib/kubelet/pods/d38cebf3-bf9a-482a-88f1-73638880723b/volumes" Jul 29 15:16:38.904572 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:38.904535 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:16:39.005274 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.005238 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:16:39.105500 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.105460 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:16:39.194879 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.194790 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:16:39.426127 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.426091 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:16:39.554340 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.554308 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d3e823-gq82b"] Jul 29 15:16:39.554745 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.554734 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b995fd88-9940-447f-b7b9-c2ae59167eaa" containerName="payload-pre-processing" Jul 29 15:16:39.554797 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.554747 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b995fd88-9940-447f-b7b9-c2ae59167eaa" containerName="payload-pre-processing" Jul 29 15:16:39.554797 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.554762 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d38cebf3-bf9a-482a-88f1-73638880723b" containerName="payload-processing" Jul 29 15:16:39.554797 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.554769 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d38cebf3-bf9a-482a-88f1-73638880723b" containerName="payload-processing" Jul 29 15:16:39.554797 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.554784 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00af3b81-cdde-4567-9e57-94bc92d5b90f" containerName="revoke-keys" Jul 29 15:16:39.554797 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.554791 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="00af3b81-cdde-4567-9e57-94bc92d5b90f" containerName="revoke-keys" Jul 29 15:16:39.554952 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.554859 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d38cebf3-bf9a-482a-88f1-73638880723b" containerName="payload-processing" Jul 29 15:16:39.554952 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.554870 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="00af3b81-cdde-4567-9e57-94bc92d5b90f" containerName="revoke-keys" Jul 29 15:16:39.554952 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.554877 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b995fd88-9940-447f-b7b9-c2ae59167eaa" containerName="payload-pre-processing" Jul 29 15:16:39.559550 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.559530 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d3e823-gq82b" Jul 29 15:16:39.561972 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.561926 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mmt6p\"" Jul 29 15:16:39.562243 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.562032 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 29 15:16:39.564085 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.564062 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d3e823-gq82b"] Jul 29 15:16:39.706647 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.706607 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc9acdf7-5829-4f16-8c9d-0223020f79ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-d3e823-gq82b\" (UID: \"cc9acdf7-5829-4f16-8c9d-0223020f79ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d3e823-gq82b" Jul 29 15:16:39.807860 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.807703 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc9acdf7-5829-4f16-8c9d-0223020f79ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-d3e823-gq82b\" (UID: \"cc9acdf7-5829-4f16-8c9d-0223020f79ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d3e823-gq82b" Jul 29 15:16:39.808479 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.808454 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc9acdf7-5829-4f16-8c9d-0223020f79ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-d3e823-gq82b\" (UID: \"cc9acdf7-5829-4f16-8c9d-0223020f79ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d3e823-gq82b" Jul 29 15:16:39.872622 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:39.872584 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d3e823-gq82b" Jul 29 15:16:40.010427 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:40.010397 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d3e823-gq82b"] Jul 29 15:16:40.012441 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:16:40.012408 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc9acdf7_5829_4f16_8c9d_0223020f79ec.slice/crio-e8007596511c7947cf8e977c96c9e070df66ed702e83242c340f85687747eccf WatchSource:0}: Error finding container e8007596511c7947cf8e977c96c9e070df66ed702e83242c340f85687747eccf: Status 404 returned error can't find the container with id e8007596511c7947cf8e977c96c9e070df66ed702e83242c340f85687747eccf Jul 29 15:16:40.863955 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:40.863918 2580 generic.go:356] "Generic (PLEG): container finished" podID="cc9acdf7-5829-4f16-8c9d-0223020f79ec" containerID="9b2e42eb361500ff2e9e5a093c96b037afbf5dd9cf7db1e7cc3772d34a7d7b9b" exitCode=0 Jul 29 15:16:40.864144 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:40.864003 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d3e823-gq82b" event={"ID":"cc9acdf7-5829-4f16-8c9d-0223020f79ec","Type":"ContainerDied","Data":"9b2e42eb361500ff2e9e5a093c96b037afbf5dd9cf7db1e7cc3772d34a7d7b9b"} Jul 29 15:16:40.864144 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:40.864064 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d3e823-gq82b" event={"ID":"cc9acdf7-5829-4f16-8c9d-0223020f79ec","Type":"ContainerStarted","Data":"e8007596511c7947cf8e977c96c9e070df66ed702e83242c340f85687747eccf"} Jul 29 15:16:42.010818 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:42.010791 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d3e823-gq82b" Jul 29 15:16:42.127914 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:42.127823 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc9acdf7-5829-4f16-8c9d-0223020f79ec-maas-api-service-ca\") pod \"cc9acdf7-5829-4f16-8c9d-0223020f79ec\" (UID: \"cc9acdf7-5829-4f16-8c9d-0223020f79ec\") " Jul 29 15:16:42.128231 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:42.128197 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc9acdf7-5829-4f16-8c9d-0223020f79ec-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cc9acdf7-5829-4f16-8c9d-0223020f79ec" (UID: "cc9acdf7-5829-4f16-8c9d-0223020f79ec"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:16:42.229070 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:42.229014 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc9acdf7-5829-4f16-8c9d-0223020f79ec-maas-api-service-ca\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:16:42.876308 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:42.876277 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d3e823-gq82b" Jul 29 15:16:42.876471 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:42.876277 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d3e823-gq82b" event={"ID":"cc9acdf7-5829-4f16-8c9d-0223020f79ec","Type":"ContainerDied","Data":"e8007596511c7947cf8e977c96c9e070df66ed702e83242c340f85687747eccf"} Jul 29 15:16:42.876471 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:42.876404 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8007596511c7947cf8e977c96c9e070df66ed702e83242c340f85687747eccf" Jul 29 15:16:55.136412 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.136377 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:16:55.170831 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.170796 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww"] Jul 29 15:16:55.171046 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.171016 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww" podUID="11b8fc23-d44f-4660-91ad-a9036a935d68" containerName="payload-processing" containerID="cri-o://4e16fa4d11683230cbaf5e68b0bfab7f3e376736f709c142987f243b50ea7e9e" gracePeriod=30 Jul 29 15:16:55.177093 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.177060 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv"] Jul 29 15:16:55.177333 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.177307 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv" podUID="5403194b-dbe8-4e8b-961b-b71cc153f767" containerName="payload-pre-processing" containerID="cri-o://5089d4b2e82ef6f979db129c849de753d2a4747877b82b7de7c2c57daed41965" gracePeriod=30 Jul 29 15:16:55.224481 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.224447 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:16:55.287846 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.287810 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:16:55.329681 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.329521 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d3e823-gq82b"] Jul 29 15:16:55.332390 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.332342 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-d3e823-gq82b"] Jul 29 15:16:55.348366 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.347658 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:16:55.464744 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.462101 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww" Jul 29 15:16:55.486611 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.486582 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv" Jul 29 15:16:55.562296 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.562260 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6b8lf\" (UniqueName: \"kubernetes.io/projected/11b8fc23-d44f-4660-91ad-a9036a935d68-kube-api-access-6b8lf\") pod \"11b8fc23-d44f-4660-91ad-a9036a935d68\" (UID: \"11b8fc23-d44f-4660-91ad-a9036a935d68\") " Jul 29 15:16:55.562486 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.562310 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11b8fc23-d44f-4660-91ad-a9036a935d68-plugins-config-volume\") pod \"11b8fc23-d44f-4660-91ad-a9036a935d68\" (UID: \"11b8fc23-d44f-4660-91ad-a9036a935d68\") " Jul 29 15:16:55.562486 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.562369 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhmc9\" (UniqueName: \"kubernetes.io/projected/5403194b-dbe8-4e8b-961b-b71cc153f767-kube-api-access-vhmc9\") pod \"5403194b-dbe8-4e8b-961b-b71cc153f767\" (UID: \"5403194b-dbe8-4e8b-961b-b71cc153f767\") " Jul 29 15:16:55.562486 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.562400 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5403194b-dbe8-4e8b-961b-b71cc153f767-plugins-config-volume\") pod \"5403194b-dbe8-4e8b-961b-b71cc153f767\" (UID: \"5403194b-dbe8-4e8b-961b-b71cc153f767\") " Jul 29 15:16:55.562810 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.562788 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5403194b-dbe8-4e8b-961b-b71cc153f767-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5403194b-dbe8-4e8b-961b-b71cc153f767" (UID: "5403194b-dbe8-4e8b-961b-b71cc153f767"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:16:55.562810 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.562796 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11b8fc23-d44f-4660-91ad-a9036a935d68-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "11b8fc23-d44f-4660-91ad-a9036a935d68" (UID: "11b8fc23-d44f-4660-91ad-a9036a935d68"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:16:55.564405 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.564384 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11b8fc23-d44f-4660-91ad-a9036a935d68-kube-api-access-6b8lf" (OuterVolumeSpecName: "kube-api-access-6b8lf") pod "11b8fc23-d44f-4660-91ad-a9036a935d68" (UID: "11b8fc23-d44f-4660-91ad-a9036a935d68"). InnerVolumeSpecName "kube-api-access-6b8lf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:16:55.564469 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.564406 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5403194b-dbe8-4e8b-961b-b71cc153f767-kube-api-access-vhmc9" (OuterVolumeSpecName: "kube-api-access-vhmc9") pod "5403194b-dbe8-4e8b-961b-b71cc153f767" (UID: "5403194b-dbe8-4e8b-961b-b71cc153f767"). InnerVolumeSpecName "kube-api-access-vhmc9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:16:55.663456 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.663421 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhmc9\" (UniqueName: \"kubernetes.io/projected/5403194b-dbe8-4e8b-961b-b71cc153f767-kube-api-access-vhmc9\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:16:55.663456 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.663452 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5403194b-dbe8-4e8b-961b-b71cc153f767-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:16:55.663456 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.663462 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6b8lf\" (UniqueName: \"kubernetes.io/projected/11b8fc23-d44f-4660-91ad-a9036a935d68-kube-api-access-6b8lf\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:16:55.663456 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.663471 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11b8fc23-d44f-4660-91ad-a9036a935d68-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:16:55.758205 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.758168 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:16:55.818718 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.818686 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc9acdf7-5829-4f16-8c9d-0223020f79ec" path="/var/lib/kubelet/pods/cc9acdf7-5829-4f16-8c9d-0223020f79ec/volumes" Jul 29 15:16:55.950602 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.950562 2580 generic.go:356] "Generic (PLEG): container finished" podID="5403194b-dbe8-4e8b-961b-b71cc153f767" containerID="5089d4b2e82ef6f979db129c849de753d2a4747877b82b7de7c2c57daed41965" exitCode=0 Jul 29 15:16:55.950832 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.950611 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv" Jul 29 15:16:55.950832 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.950655 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv" event={"ID":"5403194b-dbe8-4e8b-961b-b71cc153f767","Type":"ContainerDied","Data":"5089d4b2e82ef6f979db129c849de753d2a4747877b82b7de7c2c57daed41965"} Jul 29 15:16:55.950832 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.950752 2580 scope.go:117] "RemoveContainer" containerID="5089d4b2e82ef6f979db129c849de753d2a4747877b82b7de7c2c57daed41965" Jul 29 15:16:55.951029 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.950898 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv" event={"ID":"5403194b-dbe8-4e8b-961b-b71cc153f767","Type":"ContainerDied","Data":"e34628663506c4fefddf92133c1a2947481a6fb6881c255fcc28f8b6bf758830"} Jul 29 15:16:55.952537 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.952511 2580 generic.go:356] "Generic (PLEG): container finished" podID="11b8fc23-d44f-4660-91ad-a9036a935d68" containerID="4e16fa4d11683230cbaf5e68b0bfab7f3e376736f709c142987f243b50ea7e9e" exitCode=0 Jul 29 15:16:55.952673 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.952560 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww" event={"ID":"11b8fc23-d44f-4660-91ad-a9036a935d68","Type":"ContainerDied","Data":"4e16fa4d11683230cbaf5e68b0bfab7f3e376736f709c142987f243b50ea7e9e"} Jul 29 15:16:55.952673 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.952601 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww" event={"ID":"11b8fc23-d44f-4660-91ad-a9036a935d68","Type":"ContainerDied","Data":"09fdb7340180ce20010988258aa67a72260cb2c4e5f6fce6a3fb0b415b74a18c"} Jul 29 15:16:55.952673 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.952667 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww" Jul 29 15:16:55.961345 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.961327 2580 scope.go:117] "RemoveContainer" containerID="5089d4b2e82ef6f979db129c849de753d2a4747877b82b7de7c2c57daed41965" Jul 29 15:16:55.961664 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:16:55.961645 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5089d4b2e82ef6f979db129c849de753d2a4747877b82b7de7c2c57daed41965\": container with ID starting with 5089d4b2e82ef6f979db129c849de753d2a4747877b82b7de7c2c57daed41965 not found: ID does not exist" containerID="5089d4b2e82ef6f979db129c849de753d2a4747877b82b7de7c2c57daed41965" Jul 29 15:16:55.961774 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.961676 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5089d4b2e82ef6f979db129c849de753d2a4747877b82b7de7c2c57daed41965"} err="failed to get container status \"5089d4b2e82ef6f979db129c849de753d2a4747877b82b7de7c2c57daed41965\": rpc error: code = NotFound desc = could not find container \"5089d4b2e82ef6f979db129c849de753d2a4747877b82b7de7c2c57daed41965\": container with ID starting with 5089d4b2e82ef6f979db129c849de753d2a4747877b82b7de7c2c57daed41965 not found: ID does not exist" Jul 29 15:16:55.961774 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.961702 2580 scope.go:117] "RemoveContainer" containerID="4e16fa4d11683230cbaf5e68b0bfab7f3e376736f709c142987f243b50ea7e9e" Jul 29 15:16:55.970366 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.970345 2580 scope.go:117] "RemoveContainer" containerID="4e16fa4d11683230cbaf5e68b0bfab7f3e376736f709c142987f243b50ea7e9e" Jul 29 15:16:55.980224 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:16:55.970873 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e16fa4d11683230cbaf5e68b0bfab7f3e376736f709c142987f243b50ea7e9e\": container with ID starting with 4e16fa4d11683230cbaf5e68b0bfab7f3e376736f709c142987f243b50ea7e9e not found: ID does not exist" containerID="4e16fa4d11683230cbaf5e68b0bfab7f3e376736f709c142987f243b50ea7e9e" Jul 29 15:16:55.980224 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.970922 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e16fa4d11683230cbaf5e68b0bfab7f3e376736f709c142987f243b50ea7e9e"} err="failed to get container status \"4e16fa4d11683230cbaf5e68b0bfab7f3e376736f709c142987f243b50ea7e9e\": rpc error: code = NotFound desc = could not find container \"4e16fa4d11683230cbaf5e68b0bfab7f3e376736f709c142987f243b50ea7e9e\": container with ID starting with 4e16fa4d11683230cbaf5e68b0bfab7f3e376736f709c142987f243b50ea7e9e not found: ID does not exist" Jul 29 15:16:55.980224 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.980107 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww"] Jul 29 15:16:55.983404 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.983379 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-d3e823-8b7787584-vn8ww"] Jul 29 15:16:55.989468 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.989438 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv"] Jul 29 15:16:55.991647 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:55.991627 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d3e823-7898676cb-vdbwv"] Jul 29 15:16:57.817883 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:57.817849 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11b8fc23-d44f-4660-91ad-a9036a935d68" path="/var/lib/kubelet/pods/11b8fc23-d44f-4660-91ad-a9036a935d68/volumes" Jul 29 15:16:57.818264 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:16:57.818210 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5403194b-dbe8-4e8b-961b-b71cc153f767" path="/var/lib/kubelet/pods/5403194b-dbe8-4e8b-961b-b71cc153f767/volumes" Jul 29 15:17:01.618487 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:01.618449 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:01.725921 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:01.725872 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:01.935390 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:01.935291 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:02.179144 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:02.179096 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:02.567858 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:02.567824 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:02.890223 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:02.890138 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:03.075044 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:03.075007 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:05.350740 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:05.350693 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:05.426698 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:05.426659 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:05.520586 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:05.520546 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:05.784033 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:05.783998 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:05.939480 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:05.939450 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:06.039656 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.039574 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:06.052650 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.052618 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q"] Jul 29 15:17:06.053015 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.053002 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11b8fc23-d44f-4660-91ad-a9036a935d68" containerName="payload-processing" Jul 29 15:17:06.053071 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.053016 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="11b8fc23-d44f-4660-91ad-a9036a935d68" containerName="payload-processing" Jul 29 15:17:06.053071 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.053052 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5403194b-dbe8-4e8b-961b-b71cc153f767" containerName="payload-pre-processing" Jul 29 15:17:06.053071 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.053058 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5403194b-dbe8-4e8b-961b-b71cc153f767" containerName="payload-pre-processing" Jul 29 15:17:06.053071 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.053067 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc9acdf7-5829-4f16-8c9d-0223020f79ec" containerName="revoke-keys" Jul 29 15:17:06.053191 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.053072 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc9acdf7-5829-4f16-8c9d-0223020f79ec" containerName="revoke-keys" Jul 29 15:17:06.053191 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.053127 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="11b8fc23-d44f-4660-91ad-a9036a935d68" containerName="payload-processing" Jul 29 15:17:06.053191 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.053135 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5403194b-dbe8-4e8b-961b-b71cc153f767" containerName="payload-pre-processing" Jul 29 15:17:06.053191 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.053142 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc9acdf7-5829-4f16-8c9d-0223020f79ec" containerName="revoke-keys" Jul 29 15:17:06.056160 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.056142 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q" Jul 29 15:17:06.058438 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.058416 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-c73214-dockercfg-dbr8n\"" Jul 29 15:17:06.058655 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.058641 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-c73214\"" Jul 29 15:17:06.059517 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.059493 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf"] Jul 29 15:17:06.063291 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.063038 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf" Jul 29 15:17:06.067680 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.067648 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q"] Jul 29 15:17:06.084786 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.084750 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf"] Jul 29 15:17:06.159569 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.159528 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d2fd091-10c2-4116-84ab-5851880c637f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf\" (UID: \"5d2fd091-10c2-4116-84ab-5851880c637f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf" Jul 29 15:17:06.159569 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.159570 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-282jp\" (UniqueName: \"kubernetes.io/projected/938d357a-10d7-4e12-9e12-44db76aaa763-kube-api-access-282jp\") pod \"payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q\" (UID: \"938d357a-10d7-4e12-9e12-44db76aaa763\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q" Jul 29 15:17:06.159895 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.159597 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/938d357a-10d7-4e12-9e12-44db76aaa763-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q\" (UID: \"938d357a-10d7-4e12-9e12-44db76aaa763\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q" Jul 29 15:17:06.159895 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.159619 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdqqq\" (UniqueName: \"kubernetes.io/projected/5d2fd091-10c2-4116-84ab-5851880c637f-kube-api-access-qdqqq\") pod \"payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf\" (UID: \"5d2fd091-10c2-4116-84ab-5851880c637f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf" Jul 29 15:17:06.260788 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.260754 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/938d357a-10d7-4e12-9e12-44db76aaa763-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q\" (UID: \"938d357a-10d7-4e12-9e12-44db76aaa763\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q" Jul 29 15:17:06.260977 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.260792 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdqqq\" (UniqueName: \"kubernetes.io/projected/5d2fd091-10c2-4116-84ab-5851880c637f-kube-api-access-qdqqq\") pod \"payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf\" (UID: \"5d2fd091-10c2-4116-84ab-5851880c637f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf" Jul 29 15:17:06.260977 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.260940 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d2fd091-10c2-4116-84ab-5851880c637f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf\" (UID: \"5d2fd091-10c2-4116-84ab-5851880c637f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf" Jul 29 15:17:06.260977 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.260967 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-282jp\" (UniqueName: \"kubernetes.io/projected/938d357a-10d7-4e12-9e12-44db76aaa763-kube-api-access-282jp\") pod \"payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q\" (UID: \"938d357a-10d7-4e12-9e12-44db76aaa763\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q" Jul 29 15:17:06.261570 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.261552 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/938d357a-10d7-4e12-9e12-44db76aaa763-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q\" (UID: \"938d357a-10d7-4e12-9e12-44db76aaa763\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q" Jul 29 15:17:06.261613 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.261593 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d2fd091-10c2-4116-84ab-5851880c637f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf\" (UID: \"5d2fd091-10c2-4116-84ab-5851880c637f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf" Jul 29 15:17:06.268507 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.268484 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdqqq\" (UniqueName: \"kubernetes.io/projected/5d2fd091-10c2-4116-84ab-5851880c637f-kube-api-access-qdqqq\") pod \"payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf\" (UID: \"5d2fd091-10c2-4116-84ab-5851880c637f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf" Jul 29 15:17:06.268629 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.268609 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-282jp\" (UniqueName: \"kubernetes.io/projected/938d357a-10d7-4e12-9e12-44db76aaa763-kube-api-access-282jp\") pod \"payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q\" (UID: \"938d357a-10d7-4e12-9e12-44db76aaa763\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q" Jul 29 15:17:06.368389 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.368299 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q" Jul 29 15:17:06.376623 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.376592 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf" Jul 29 15:17:06.520179 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.520150 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q"] Jul 29 15:17:06.523406 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:17:06.523368 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod938d357a_10d7_4e12_9e12_44db76aaa763.slice/crio-888d785c2a3885ebad5c7476f5ba859315a104633ba2a670b1ba30785231a4dc WatchSource:0}: Error finding container 888d785c2a3885ebad5c7476f5ba859315a104633ba2a670b1ba30785231a4dc: Status 404 returned error can't find the container with id 888d785c2a3885ebad5c7476f5ba859315a104633ba2a670b1ba30785231a4dc Jul 29 15:17:06.749236 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:06.749205 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf"] Jul 29 15:17:06.752211 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:17:06.752176 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d2fd091_10c2_4116_84ab_5851880c637f.slice/crio-b45d1360c775fa8eff447dde9e3bb509ad0f23c2511a3114087f780345f7c5ab WatchSource:0}: Error finding container b45d1360c775fa8eff447dde9e3bb509ad0f23c2511a3114087f780345f7c5ab: Status 404 returned error can't find the container with id b45d1360c775fa8eff447dde9e3bb509ad0f23c2511a3114087f780345f7c5ab Jul 29 15:17:07.029061 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:07.028941 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf" event={"ID":"5d2fd091-10c2-4116-84ab-5851880c637f","Type":"ContainerStarted","Data":"4f93d823a3eef00ef4729b6e3739b586c4780d36645cc889c5c7e58890d71076"} Jul 29 15:17:07.029061 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:07.029001 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf" event={"ID":"5d2fd091-10c2-4116-84ab-5851880c637f","Type":"ContainerStarted","Data":"b45d1360c775fa8eff447dde9e3bb509ad0f23c2511a3114087f780345f7c5ab"} Jul 29 15:17:07.029061 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:07.029057 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf" Jul 29 15:17:07.030824 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:07.030755 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q" event={"ID":"938d357a-10d7-4e12-9e12-44db76aaa763","Type":"ContainerStarted","Data":"74191fc531a9e6ee6bdc73ab874c93c70ebe04d9df4fac6cdcd5a99df3cd4d8b"} Jul 29 15:17:07.030824 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:07.030796 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q" event={"ID":"938d357a-10d7-4e12-9e12-44db76aaa763","Type":"ContainerStarted","Data":"888d785c2a3885ebad5c7476f5ba859315a104633ba2a670b1ba30785231a4dc"} Jul 29 15:17:07.031014 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:07.030838 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q" Jul 29 15:17:07.045537 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:07.045493 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf" podStartSLOduration=1.045478655 podStartE2EDuration="1.045478655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:17:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:17:07.044364398 +0000 UTC m=+2503.772513588" watchObservedRunningTime="2026-07-29 15:17:07.045478655 +0000 UTC m=+2503.773627843" Jul 29 15:17:07.061767 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:07.061700 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q" podStartSLOduration=1.061681608 podStartE2EDuration="1.061681608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:17:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:17:07.061108625 +0000 UTC m=+2503.789257817" watchObservedRunningTime="2026-07-29 15:17:07.061681608 +0000 UTC m=+2503.789830800" Jul 29 15:17:18.039413 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:18.039333 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q" Jul 29 15:17:18.039906 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:18.039497 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf" Jul 29 15:17:30.434174 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:30.434132 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:30.797841 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:30.797814 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:31.129053 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:31.128949 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:31.612212 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:31.612180 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:34.542157 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:34.542089 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:34.679800 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:34.679760 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:35.431580 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.431543 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d"] Jul 29 15:17:35.435166 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.435137 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d" Jul 29 15:17:35.437583 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.437564 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-93e799\"" Jul 29 15:17:35.446247 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.446223 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d"] Jul 29 15:17:35.457351 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.457328 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8"] Jul 29 15:17:35.461015 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.460993 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8" Jul 29 15:17:35.463202 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.463178 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-93e799-dockercfg-89pd9\"" Jul 29 15:17:35.472594 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.472575 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8"] Jul 29 15:17:35.517657 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.517637 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14f15644-65fc-43d8-9e52-b6ee74ef108d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8\" (UID: \"14f15644-65fc-43d8-9e52-b6ee74ef108d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8" Jul 29 15:17:35.517801 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.517687 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wphk2\" (UniqueName: \"kubernetes.io/projected/14f15644-65fc-43d8-9e52-b6ee74ef108d-kube-api-access-wphk2\") pod \"payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8\" (UID: \"14f15644-65fc-43d8-9e52-b6ee74ef108d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8" Jul 29 15:17:35.517801 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.517705 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80d9f2c0-5975-4b05-8413-8a324b252448-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d\" (UID: \"80d9f2c0-5975-4b05-8413-8a324b252448\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d" Jul 29 15:17:35.517801 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.517777 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sn2k\" (UniqueName: \"kubernetes.io/projected/80d9f2c0-5975-4b05-8413-8a324b252448-kube-api-access-9sn2k\") pod \"payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d\" (UID: \"80d9f2c0-5975-4b05-8413-8a324b252448\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d" Jul 29 15:17:35.618889 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.618848 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14f15644-65fc-43d8-9e52-b6ee74ef108d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8\" (UID: \"14f15644-65fc-43d8-9e52-b6ee74ef108d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8" Jul 29 15:17:35.619325 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.618936 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wphk2\" (UniqueName: \"kubernetes.io/projected/14f15644-65fc-43d8-9e52-b6ee74ef108d-kube-api-access-wphk2\") pod \"payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8\" (UID: \"14f15644-65fc-43d8-9e52-b6ee74ef108d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8" Jul 29 15:17:35.619325 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.618965 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80d9f2c0-5975-4b05-8413-8a324b252448-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d\" (UID: \"80d9f2c0-5975-4b05-8413-8a324b252448\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d" Jul 29 15:17:35.619325 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.619007 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sn2k\" (UniqueName: \"kubernetes.io/projected/80d9f2c0-5975-4b05-8413-8a324b252448-kube-api-access-9sn2k\") pod \"payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d\" (UID: \"80d9f2c0-5975-4b05-8413-8a324b252448\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d" Jul 29 15:17:35.619611 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.619591 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14f15644-65fc-43d8-9e52-b6ee74ef108d-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8\" (UID: \"14f15644-65fc-43d8-9e52-b6ee74ef108d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8" Jul 29 15:17:35.619679 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.619661 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80d9f2c0-5975-4b05-8413-8a324b252448-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d\" (UID: \"80d9f2c0-5975-4b05-8413-8a324b252448\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d" Jul 29 15:17:35.627046 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.627023 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wphk2\" (UniqueName: \"kubernetes.io/projected/14f15644-65fc-43d8-9e52-b6ee74ef108d-kube-api-access-wphk2\") pod \"payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8\" (UID: \"14f15644-65fc-43d8-9e52-b6ee74ef108d\") " pod="openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8" Jul 29 15:17:35.627175 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.627134 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sn2k\" (UniqueName: \"kubernetes.io/projected/80d9f2c0-5975-4b05-8413-8a324b252448-kube-api-access-9sn2k\") pod \"payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d\" (UID: \"80d9f2c0-5975-4b05-8413-8a324b252448\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d" Jul 29 15:17:35.747405 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.747374 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d" Jul 29 15:17:35.771838 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.771809 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8" Jul 29 15:17:35.899004 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.898202 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:35.899629 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.899600 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d"] Jul 29 15:17:35.901533 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:17:35.901508 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80d9f2c0_5975_4b05_8413_8a324b252448.slice/crio-08c217217e3a4c92fd479cf3bbe13739a57c80e1a09110a3400d297275b38415 WatchSource:0}: Error finding container 08c217217e3a4c92fd479cf3bbe13739a57c80e1a09110a3400d297275b38415: Status 404 returned error can't find the container with id 08c217217e3a4c92fd479cf3bbe13739a57c80e1a09110a3400d297275b38415 Jul 29 15:17:35.922374 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.922351 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8"] Jul 29 15:17:35.924434 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:17:35.924406 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14f15644_65fc_43d8_9e52_b6ee74ef108d.slice/crio-1efe4e06cbdcd7d81da3a4c3b3dbbd99d7a5429cbd06a0edbe98d8d176de5200 WatchSource:0}: Error finding container 1efe4e06cbdcd7d81da3a4c3b3dbbd99d7a5429cbd06a0edbe98d8d176de5200: Status 404 returned error can't find the container with id 1efe4e06cbdcd7d81da3a4c3b3dbbd99d7a5429cbd06a0edbe98d8d176de5200 Jul 29 15:17:35.994402 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:35.994371 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:36.203070 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:36.202980 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d" event={"ID":"80d9f2c0-5975-4b05-8413-8a324b252448","Type":"ContainerStarted","Data":"ccf156d7a518d8007523060d80dfb470afa7602affd9dc3320f653845150ff37"} Jul 29 15:17:36.203070 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:36.203038 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d" event={"ID":"80d9f2c0-5975-4b05-8413-8a324b252448","Type":"ContainerStarted","Data":"08c217217e3a4c92fd479cf3bbe13739a57c80e1a09110a3400d297275b38415"} Jul 29 15:17:36.203301 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:36.203173 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d" Jul 29 15:17:36.204623 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:36.204598 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8" event={"ID":"14f15644-65fc-43d8-9e52-b6ee74ef108d","Type":"ContainerStarted","Data":"bc55dd0b45c37e291b8d0940db0cc16d2b315a0a3c1deb720ffe4eb211ba1295"} Jul 29 15:17:36.204750 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:36.204640 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8" Jul 29 15:17:36.204750 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:36.204664 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8" event={"ID":"14f15644-65fc-43d8-9e52-b6ee74ef108d","Type":"ContainerStarted","Data":"1efe4e06cbdcd7d81da3a4c3b3dbbd99d7a5429cbd06a0edbe98d8d176de5200"} Jul 29 15:17:36.236311 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:36.236266 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d" podStartSLOduration=1.236254065 podStartE2EDuration="1.236254065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:17:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:17:36.234896105 +0000 UTC m=+2532.963045294" watchObservedRunningTime="2026-07-29 15:17:36.236254065 +0000 UTC m=+2532.964403256" Jul 29 15:17:36.254250 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:36.254202 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8" podStartSLOduration=1.2541836 podStartE2EDuration="1.2541836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:17:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:17:36.25217486 +0000 UTC m=+2532.980324051" watchObservedRunningTime="2026-07-29 15:17:36.2541836 +0000 UTC m=+2532.982332793" Jul 29 15:17:47.212050 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:47.212020 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d" Jul 29 15:17:47.212525 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:47.212204 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8" Jul 29 15:17:52.036420 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:52.036385 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:57.151551 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:57.151517 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:57.254736 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:57.254679 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:17:59.635929 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:17:59.635891 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:18:13.829126 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:18:13.829089 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:18:26.875247 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:18:26.875214 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:19:14.037444 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:19:14.037363 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:19:40.295853 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:19:40.295817 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:20:23.966949 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:20:23.966924 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 15:20:23.975005 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:20:23.974983 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 15:21:24.129844 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:21:24.129815 2580 scope.go:117] "RemoveContainer" containerID="65bbfb99a173dd13eb7fb52790bcbae004fed166d369e041b42faed0d16d5d55" Jul 29 15:21:24.138211 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:21:24.138193 2580 scope.go:117] "RemoveContainer" containerID="8f8d5f4937ce37023871a5665eeafe5279acbcccbf20ab6426080aed5d4d1bf6" Jul 29 15:21:24.145764 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:21:24.145749 2580 scope.go:117] "RemoveContainer" containerID="94347ffef05968f5372e2d8bda7c9426aab5503dfaaee28f0dbbef123984060f" Jul 29 15:22:24.173769 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:22:24.173717 2580 scope.go:117] "RemoveContainer" containerID="bad7c29648a2ba606f77da22d3bfab4a59317e520fa728f3e3dcba9c47b77f51" Jul 29 15:22:57.587075 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:22:57.587044 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:23:07.033030 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:23:07.032994 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:23:07.911133 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:23:07.911097 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:23:10.258283 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:23:10.258249 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:23:10.397854 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:23:10.397824 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:23:22.742454 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:23:22.742423 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:23:24.200741 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:23:24.200704 2580 scope.go:117] "RemoveContainer" containerID="9b2e42eb361500ff2e9e5a093c96b037afbf5dd9cf7db1e7cc3772d34a7d7b9b" Jul 29 15:24:02.949280 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:24:02.949246 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:24:10.040280 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:24:10.040220 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:24:10.091445 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:24:10.091415 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:25:01.313335 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:25:01.313231 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:25:08.694227 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:25:08.694179 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:25:09.637321 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:25:09.637290 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:25:14.584987 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:25:14.584951 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:25:16.395034 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:25:16.395000 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:25:23.999952 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:25:23.999927 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 15:25:24.008705 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:25:24.008684 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 15:25:35.567747 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:25:35.567695 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:25:47.167871 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:25:47.167824 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:26:53.929094 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:53.929057 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:26:54.743873 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:54.743837 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:26:55.702080 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:55.702053 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7695-140291048021680"] Jul 29 15:26:55.705049 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:55.705018 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7695-140291048021680" Jul 29 15:26:55.706909 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:55.706888 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-n6sn7\"" Jul 29 15:26:55.713169 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:55.713148 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7695-140291048021680"] Jul 29 15:26:55.779756 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:55.779718 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xzqx\" (UniqueName: \"kubernetes.io/projected/c97cbf71-3325-46bf-813d-84bfa812f77d-kube-api-access-4xzqx\") pod \"test-curl-7695-140291048021680\" (UID: \"c97cbf71-3325-46bf-813d-84bfa812f77d\") " pod="opendatahub/test-curl-7695-140291048021680" Jul 29 15:26:55.880682 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:55.880660 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xzqx\" (UniqueName: \"kubernetes.io/projected/c97cbf71-3325-46bf-813d-84bfa812f77d-kube-api-access-4xzqx\") pod \"test-curl-7695-140291048021680\" (UID: \"c97cbf71-3325-46bf-813d-84bfa812f77d\") " pod="opendatahub/test-curl-7695-140291048021680" Jul 29 15:26:55.888390 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:55.888368 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xzqx\" (UniqueName: \"kubernetes.io/projected/c97cbf71-3325-46bf-813d-84bfa812f77d-kube-api-access-4xzqx\") pod \"test-curl-7695-140291048021680\" (UID: \"c97cbf71-3325-46bf-813d-84bfa812f77d\") " pod="opendatahub/test-curl-7695-140291048021680" Jul 29 15:26:56.014564 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:56.014540 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7695-140291048021680" Jul 29 15:26:56.141178 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:56.141152 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7695-140291048021680"] Jul 29 15:26:56.142735 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:26:56.142688 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc97cbf71_3325_46bf_813d_84bfa812f77d.slice/crio-654663e2eb45e5132c301d9abe51222d5ff3c9c5376b2de6e6bac3d4e05fa02f WatchSource:0}: Error finding container 654663e2eb45e5132c301d9abe51222d5ff3c9c5376b2de6e6bac3d4e05fa02f: Status 404 returned error can't find the container with id 654663e2eb45e5132c301d9abe51222d5ff3c9c5376b2de6e6bac3d4e05fa02f Jul 29 15:26:56.144068 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:56.144050 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 15:26:56.326206 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:56.326149 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7695-140291048021680" event={"ID":"c97cbf71-3325-46bf-813d-84bfa812f77d","Type":"ContainerStarted","Data":"654663e2eb45e5132c301d9abe51222d5ff3c9c5376b2de6e6bac3d4e05fa02f"} Jul 29 15:26:57.332912 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:57.332879 2580 generic.go:356] "Generic (PLEG): container finished" podID="c97cbf71-3325-46bf-813d-84bfa812f77d" containerID="0b0527da6413807226c7a18f8cc4ce071d9eb994cdb342838c9f4777c21548a1" exitCode=0 Jul 29 15:26:57.333271 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:57.332930 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7695-140291048021680" event={"ID":"c97cbf71-3325-46bf-813d-84bfa812f77d","Type":"ContainerDied","Data":"0b0527da6413807226c7a18f8cc4ce071d9eb994cdb342838c9f4777c21548a1"} Jul 29 15:26:58.480046 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:58.480023 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7695-140291048021680" Jul 29 15:26:58.498850 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:58.498828 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xzqx\" (UniqueName: \"kubernetes.io/projected/c97cbf71-3325-46bf-813d-84bfa812f77d-kube-api-access-4xzqx\") pod \"c97cbf71-3325-46bf-813d-84bfa812f77d\" (UID: \"c97cbf71-3325-46bf-813d-84bfa812f77d\") " Jul 29 15:26:58.499201 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:58.499179 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7695-140291048021680_c97cbf71-3325-46bf-813d-84bfa812f77d/test-curl-7695-140291048021680/0.log" Jul 29 15:26:58.500836 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:58.500807 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c97cbf71-3325-46bf-813d-84bfa812f77d-kube-api-access-4xzqx" (OuterVolumeSpecName: "kube-api-access-4xzqx") pod "c97cbf71-3325-46bf-813d-84bfa812f77d" (UID: "c97cbf71-3325-46bf-813d-84bfa812f77d"). InnerVolumeSpecName "kube-api-access-4xzqx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:26:58.526605 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:58.526583 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7695-140291048021680"] Jul 29 15:26:58.531917 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:58.531895 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7695-140291048021680"] Jul 29 15:26:58.600489 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:58.600434 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xzqx\" (UniqueName: \"kubernetes.io/projected/c97cbf71-3325-46bf-813d-84bfa812f77d-kube-api-access-4xzqx\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:26:59.347198 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:59.347174 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7695-140291048021680" Jul 29 15:26:59.347353 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:59.347214 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="654663e2eb45e5132c301d9abe51222d5ff3c9c5376b2de6e6bac3d4e05fa02f" Jul 29 15:26:59.356835 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:26:59.356808 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7695-140291048021680\" is forbidden: User \"system:node:ip-10-0-132-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-42.ec2.internal' and this object" podUID="c97cbf71-3325-46bf-813d-84bfa812f77d" pod="opendatahub/test-curl-7695-140291048021680" Jul 29 15:26:59.817905 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:26:59.817883 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c97cbf71-3325-46bf-813d-84bfa812f77d" path="/var/lib/kubelet/pods/c97cbf71-3325-46bf-813d-84bfa812f77d/volumes" Jul 29 15:27:01.479437 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:01.479403 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7695-140291047942160"] Jul 29 15:27:01.479841 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:01.479826 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c97cbf71-3325-46bf-813d-84bfa812f77d" containerName="test-curl-7695-140291048021680" Jul 29 15:27:01.479841 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:01.479842 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c97cbf71-3325-46bf-813d-84bfa812f77d" containerName="test-curl-7695-140291048021680" Jul 29 15:27:01.479928 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:01.479913 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c97cbf71-3325-46bf-813d-84bfa812f77d" containerName="test-curl-7695-140291048021680" Jul 29 15:27:01.483097 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:01.483081 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7695-140291047942160" Jul 29 15:27:01.484832 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:01.484805 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-n6sn7\"" Jul 29 15:27:01.488791 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:01.488767 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7695-140291047942160"] Jul 29 15:27:01.527271 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:01.527243 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g7k2\" (UniqueName: \"kubernetes.io/projected/920e9f1e-675a-4cdf-b909-c024633b0956-kube-api-access-8g7k2\") pod \"test-curl-7695-140291047942160\" (UID: \"920e9f1e-675a-4cdf-b909-c024633b0956\") " pod="opendatahub/test-curl-7695-140291047942160" Jul 29 15:27:01.628539 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:01.628513 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g7k2\" (UniqueName: \"kubernetes.io/projected/920e9f1e-675a-4cdf-b909-c024633b0956-kube-api-access-8g7k2\") pod \"test-curl-7695-140291047942160\" (UID: \"920e9f1e-675a-4cdf-b909-c024633b0956\") " pod="opendatahub/test-curl-7695-140291047942160" Jul 29 15:27:01.636645 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:01.636625 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g7k2\" (UniqueName: \"kubernetes.io/projected/920e9f1e-675a-4cdf-b909-c024633b0956-kube-api-access-8g7k2\") pod \"test-curl-7695-140291047942160\" (UID: \"920e9f1e-675a-4cdf-b909-c024633b0956\") " pod="opendatahub/test-curl-7695-140291047942160" Jul 29 15:27:01.793438 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:01.793419 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7695-140291047942160" Jul 29 15:27:02.131554 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:02.131378 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7695-140291047942160"] Jul 29 15:27:02.134201 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:27:02.134171 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod920e9f1e_675a_4cdf_b909_c024633b0956.slice/crio-da6e1638a50ea0b1068444a3740b2765d85ee5fe981a6a5ea68d9a92d6f4a15a WatchSource:0}: Error finding container da6e1638a50ea0b1068444a3740b2765d85ee5fe981a6a5ea68d9a92d6f4a15a: Status 404 returned error can't find the container with id da6e1638a50ea0b1068444a3740b2765d85ee5fe981a6a5ea68d9a92d6f4a15a Jul 29 15:27:02.365688 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:02.365663 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7695-140291047942160" event={"ID":"920e9f1e-675a-4cdf-b909-c024633b0956","Type":"ContainerStarted","Data":"da6e1638a50ea0b1068444a3740b2765d85ee5fe981a6a5ea68d9a92d6f4a15a"} Jul 29 15:27:03.372501 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:03.372472 2580 generic.go:356] "Generic (PLEG): container finished" podID="920e9f1e-675a-4cdf-b909-c024633b0956" containerID="148b341b33100b87fa38e40cbfdab6128e0f6408c5ec94c6a5e692e55d5c9f8a" exitCode=0 Jul 29 15:27:03.372846 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:03.372515 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7695-140291047942160" event={"ID":"920e9f1e-675a-4cdf-b909-c024633b0956","Type":"ContainerDied","Data":"148b341b33100b87fa38e40cbfdab6128e0f6408c5ec94c6a5e692e55d5c9f8a"} Jul 29 15:27:04.510527 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:04.510506 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7695-140291047942160" Jul 29 15:27:04.530695 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:04.530674 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7695-140291047942160_920e9f1e-675a-4cdf-b909-c024633b0956/test-curl-7695-140291047942160/0.log" Jul 29 15:27:04.550109 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:04.550080 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8g7k2\" (UniqueName: \"kubernetes.io/projected/920e9f1e-675a-4cdf-b909-c024633b0956-kube-api-access-8g7k2\") pod \"920e9f1e-675a-4cdf-b909-c024633b0956\" (UID: \"920e9f1e-675a-4cdf-b909-c024633b0956\") " Jul 29 15:27:04.552471 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:04.552441 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/920e9f1e-675a-4cdf-b909-c024633b0956-kube-api-access-8g7k2" (OuterVolumeSpecName: "kube-api-access-8g7k2") pod "920e9f1e-675a-4cdf-b909-c024633b0956" (UID: "920e9f1e-675a-4cdf-b909-c024633b0956"). InnerVolumeSpecName "kube-api-access-8g7k2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:27:04.554596 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:04.554574 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7695-140291047942160"] Jul 29 15:27:04.556937 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:04.556916 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7695-140291047942160"] Jul 29 15:27:04.651039 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:04.650987 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8g7k2\" (UniqueName: \"kubernetes.io/projected/920e9f1e-675a-4cdf-b909-c024633b0956-kube-api-access-8g7k2\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:27:05.384358 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:05.384333 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7695-140291047942160" Jul 29 15:27:05.384500 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:05.384372 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da6e1638a50ea0b1068444a3740b2765d85ee5fe981a6a5ea68d9a92d6f4a15a" Jul 29 15:27:05.395803 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:27:05.395743 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7695-140291047942160\" is forbidden: User \"system:node:ip-10-0-132-42.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-42.ec2.internal' and this object" podUID="920e9f1e-675a-4cdf-b909-c024633b0956" pod="opendatahub/test-curl-7695-140291047942160" Jul 29 15:27:05.817377 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:05.817352 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="920e9f1e-675a-4cdf-b909-c024633b0956" path="/var/lib/kubelet/pods/920e9f1e-675a-4cdf-b909-c024633b0956/volumes" Jul 29 15:27:13.884701 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:13.884661 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:27:14.211228 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:14.211150 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:27:14.565408 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:14.565368 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:27:17.434652 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:17.434621 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:27:17.526024 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:17.525994 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:27:17.604692 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:17.604665 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:27:17.743190 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:17.743151 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:27:18.196641 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.196553 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:27:18.256225 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.256198 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw"] Jul 29 15:27:18.256586 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.256574 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="920e9f1e-675a-4cdf-b909-c024633b0956" containerName="test-curl-7695-140291047942160" Jul 29 15:27:18.256674 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.256588 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="920e9f1e-675a-4cdf-b909-c024633b0956" containerName="test-curl-7695-140291047942160" Jul 29 15:27:18.256674 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.256662 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="920e9f1e-675a-4cdf-b909-c024633b0956" containerName="test-curl-7695-140291047942160" Jul 29 15:27:18.261309 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.261289 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw" Jul 29 15:27:18.263308 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.263286 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-3261e1\"" Jul 29 15:27:18.271357 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.271333 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw"] Jul 29 15:27:18.284967 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.284903 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:27:18.290042 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.290021 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j"] Jul 29 15:27:18.293810 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.293790 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j" Jul 29 15:27:18.296158 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.296142 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-3261e1-dockercfg-q7j4c\"" Jul 29 15:27:18.302078 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.302060 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j"] Jul 29 15:27:18.357307 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.357282 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjzv5\" (UniqueName: \"kubernetes.io/projected/b8708b34-f25c-4494-a410-0d9c7d5ba1ee-kube-api-access-bjzv5\") pod \"payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j\" (UID: \"b8708b34-f25c-4494-a410-0d9c7d5ba1ee\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j" Jul 29 15:27:18.357414 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.357333 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0acec3d3-3ce1-4086-bec0-08be2669f967-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw\" (UID: \"0acec3d3-3ce1-4086-bec0-08be2669f967\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw" Jul 29 15:27:18.357461 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.357433 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv95g\" (UniqueName: \"kubernetes.io/projected/0acec3d3-3ce1-4086-bec0-08be2669f967-kube-api-access-cv95g\") pod \"payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw\" (UID: \"0acec3d3-3ce1-4086-bec0-08be2669f967\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw" Jul 29 15:27:18.357518 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.357502 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8708b34-f25c-4494-a410-0d9c7d5ba1ee-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j\" (UID: \"b8708b34-f25c-4494-a410-0d9c7d5ba1ee\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j" Jul 29 15:27:18.458242 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.458173 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjzv5\" (UniqueName: \"kubernetes.io/projected/b8708b34-f25c-4494-a410-0d9c7d5ba1ee-kube-api-access-bjzv5\") pod \"payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j\" (UID: \"b8708b34-f25c-4494-a410-0d9c7d5ba1ee\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j" Jul 29 15:27:18.458242 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.458221 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0acec3d3-3ce1-4086-bec0-08be2669f967-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw\" (UID: \"0acec3d3-3ce1-4086-bec0-08be2669f967\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw" Jul 29 15:27:18.458606 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.458249 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv95g\" (UniqueName: \"kubernetes.io/projected/0acec3d3-3ce1-4086-bec0-08be2669f967-kube-api-access-cv95g\") pod \"payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw\" (UID: \"0acec3d3-3ce1-4086-bec0-08be2669f967\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw" Jul 29 15:27:18.458606 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.458367 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8708b34-f25c-4494-a410-0d9c7d5ba1ee-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j\" (UID: \"b8708b34-f25c-4494-a410-0d9c7d5ba1ee\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j" Jul 29 15:27:18.458950 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.458934 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0acec3d3-3ce1-4086-bec0-08be2669f967-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw\" (UID: \"0acec3d3-3ce1-4086-bec0-08be2669f967\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw" Jul 29 15:27:18.459071 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.459054 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8708b34-f25c-4494-a410-0d9c7d5ba1ee-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j\" (UID: \"b8708b34-f25c-4494-a410-0d9c7d5ba1ee\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j" Jul 29 15:27:18.466604 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.466579 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv95g\" (UniqueName: \"kubernetes.io/projected/0acec3d3-3ce1-4086-bec0-08be2669f967-kube-api-access-cv95g\") pod \"payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw\" (UID: \"0acec3d3-3ce1-4086-bec0-08be2669f967\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw" Jul 29 15:27:18.466604 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.466601 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjzv5\" (UniqueName: \"kubernetes.io/projected/b8708b34-f25c-4494-a410-0d9c7d5ba1ee-kube-api-access-bjzv5\") pod \"payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j\" (UID: \"b8708b34-f25c-4494-a410-0d9c7d5ba1ee\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j" Jul 29 15:27:18.572281 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.572256 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw" Jul 29 15:27:18.604347 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.604324 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j" Jul 29 15:27:18.749582 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.749544 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j"] Jul 29 15:27:18.753247 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:27:18.753214 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8708b34_f25c_4494_a410_0d9c7d5ba1ee.slice/crio-8f448a3b5ef956c0a8176e1da7ec7c0f40daf393826e105445557995eaca82e9 WatchSource:0}: Error finding container 8f448a3b5ef956c0a8176e1da7ec7c0f40daf393826e105445557995eaca82e9: Status 404 returned error can't find the container with id 8f448a3b5ef956c0a8176e1da7ec7c0f40daf393826e105445557995eaca82e9 Jul 29 15:27:18.926895 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:18.926871 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw"] Jul 29 15:27:18.928709 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:27:18.928686 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0acec3d3_3ce1_4086_bec0_08be2669f967.slice/crio-2a6fd1b0c1e68eb486d3ff7c11056bef4d8926833c046f868cd624e3416f0a3c WatchSource:0}: Error finding container 2a6fd1b0c1e68eb486d3ff7c11056bef4d8926833c046f868cd624e3416f0a3c: Status 404 returned error can't find the container with id 2a6fd1b0c1e68eb486d3ff7c11056bef4d8926833c046f868cd624e3416f0a3c Jul 29 15:27:19.462187 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:19.462154 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw" event={"ID":"0acec3d3-3ce1-4086-bec0-08be2669f967","Type":"ContainerStarted","Data":"8d0bf9261a41a5f007871941aeca977dc61f423d2ea6220f58e5bbeb6c4ae22a"} Jul 29 15:27:19.462614 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:19.462217 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw" Jul 29 15:27:19.462614 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:19.462237 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw" event={"ID":"0acec3d3-3ce1-4086-bec0-08be2669f967","Type":"ContainerStarted","Data":"2a6fd1b0c1e68eb486d3ff7c11056bef4d8926833c046f868cd624e3416f0a3c"} Jul 29 15:27:19.463940 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:19.463916 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j" event={"ID":"b8708b34-f25c-4494-a410-0d9c7d5ba1ee","Type":"ContainerStarted","Data":"9cb3473cad4ca081d7fcd7b842f6ff260e855cb5a9983963588384c92210b999"} Jul 29 15:27:19.464052 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:19.463958 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j" event={"ID":"b8708b34-f25c-4494-a410-0d9c7d5ba1ee","Type":"ContainerStarted","Data":"8f448a3b5ef956c0a8176e1da7ec7c0f40daf393826e105445557995eaca82e9"} Jul 29 15:27:19.464052 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:19.463979 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j" Jul 29 15:27:19.490315 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:19.490275 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw" podStartSLOduration=1.490262952 podStartE2EDuration="1.490262952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:27:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:27:19.488068459 +0000 UTC m=+3116.216217647" watchObservedRunningTime="2026-07-29 15:27:19.490262952 +0000 UTC m=+3116.218412139" Jul 29 15:27:19.520588 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:19.520552 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j" podStartSLOduration=1.5205392039999999 podStartE2EDuration="1.520539204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:27:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:27:19.518006138 +0000 UTC m=+3116.246155343" watchObservedRunningTime="2026-07-29 15:27:19.520539204 +0000 UTC m=+3116.248688392" Jul 29 15:27:30.471615 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:30.471582 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw" Jul 29 15:27:30.472259 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:30.471782 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j" Jul 29 15:27:42.434880 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:42.434822 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:27:43.097799 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:43.097760 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:27:43.418822 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:43.418715 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:27:43.661375 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:43.661342 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fqmpx"] Jul 29 15:27:43.664593 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:43.664577 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fqmpx" Jul 29 15:27:43.667249 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:43.667232 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-qnl4h\"" Jul 29 15:27:43.673487 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:43.673467 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fqmpx"] Jul 29 15:27:43.746074 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:43.746050 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw968\" (UniqueName: \"kubernetes.io/projected/62abd7b2-d785-4360-a439-ddf537627bd6-kube-api-access-cw968\") pod \"kuadrant-operator-catalog-fqmpx\" (UID: \"62abd7b2-d785-4360-a439-ddf537627bd6\") " pod="kuadrant-system/kuadrant-operator-catalog-fqmpx" Jul 29 15:27:43.846540 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:43.846511 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw968\" (UniqueName: \"kubernetes.io/projected/62abd7b2-d785-4360-a439-ddf537627bd6-kube-api-access-cw968\") pod \"kuadrant-operator-catalog-fqmpx\" (UID: \"62abd7b2-d785-4360-a439-ddf537627bd6\") " pod="kuadrant-system/kuadrant-operator-catalog-fqmpx" Jul 29 15:27:43.846860 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:43.846831 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:27:43.856733 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:43.856702 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw968\" (UniqueName: \"kubernetes.io/projected/62abd7b2-d785-4360-a439-ddf537627bd6-kube-api-access-cw968\") pod \"kuadrant-operator-catalog-fqmpx\" (UID: \"62abd7b2-d785-4360-a439-ddf537627bd6\") " pod="kuadrant-system/kuadrant-operator-catalog-fqmpx" Jul 29 15:27:43.974316 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:43.974259 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fqmpx" Jul 29 15:27:44.104436 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:44.104411 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fqmpx"] Jul 29 15:27:44.106038 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:27:44.106005 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62abd7b2_d785_4360_a439_ddf537627bd6.slice/crio-d10a887d85a14d57ba50f36e364cd4cf0c9565070d833e0f3460a2a58d200ceb WatchSource:0}: Error finding container d10a887d85a14d57ba50f36e364cd4cf0c9565070d833e0f3460a2a58d200ceb: Status 404 returned error can't find the container with id d10a887d85a14d57ba50f36e364cd4cf0c9565070d833e0f3460a2a58d200ceb Jul 29 15:27:44.611822 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:44.611786 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fqmpx" event={"ID":"62abd7b2-d785-4360-a439-ddf537627bd6","Type":"ContainerStarted","Data":"d10a887d85a14d57ba50f36e364cd4cf0c9565070d833e0f3460a2a58d200ceb"} Jul 29 15:27:46.338758 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:46.338708 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:27:46.626135 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:46.626057 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fqmpx" event={"ID":"62abd7b2-d785-4360-a439-ddf537627bd6","Type":"ContainerStarted","Data":"9089d05c4b4a9f5aeca8c8e5d3efc7a572a8f9836d934f403d2768ae80be3a1c"} Jul 29 15:27:46.644867 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:46.644818 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-fqmpx" podStartSLOduration=2.155388334 podStartE2EDuration="3.644799911s" totalImagesPullingTime="1.489411577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:27:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:27:46.644115257 +0000 UTC m=+3143.372264445" watchObservedRunningTime="2026-07-29 15:27:46.644799911 +0000 UTC m=+3143.372949097" Jul 29 15:27:47.136922 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.136887 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt"] Jul 29 15:27:47.140640 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.140626 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt" Jul 29 15:27:47.142921 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.142898 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-a4092e\"" Jul 29 15:27:47.156648 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.156621 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt"] Jul 29 15:27:47.158989 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.158967 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj"] Jul 29 15:27:47.162515 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.162497 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj" Jul 29 15:27:47.165292 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.165276 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-a4092e-dockercfg-ptwqj\"" Jul 29 15:27:47.173318 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.173290 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj"] Jul 29 15:27:47.279702 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.279671 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9qdl\" (UniqueName: \"kubernetes.io/projected/16cf7c54-77f6-45ea-91e4-327699f19c90-kube-api-access-b9qdl\") pod \"payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt\" (UID: \"16cf7c54-77f6-45ea-91e4-327699f19c90\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt" Jul 29 15:27:47.279907 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.279712 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf4qp\" (UniqueName: \"kubernetes.io/projected/6dbe1be3-6e15-44da-9d20-6d5319dde07c-kube-api-access-tf4qp\") pod \"payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj\" (UID: \"6dbe1be3-6e15-44da-9d20-6d5319dde07c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj" Jul 29 15:27:47.279907 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.279819 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16cf7c54-77f6-45ea-91e4-327699f19c90-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt\" (UID: \"16cf7c54-77f6-45ea-91e4-327699f19c90\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt" Jul 29 15:27:47.279907 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.279885 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6dbe1be3-6e15-44da-9d20-6d5319dde07c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj\" (UID: \"6dbe1be3-6e15-44da-9d20-6d5319dde07c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj" Jul 29 15:27:47.291206 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.291166 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:27:47.355303 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.355279 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:27:47.381315 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.381288 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6dbe1be3-6e15-44da-9d20-6d5319dde07c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj\" (UID: \"6dbe1be3-6e15-44da-9d20-6d5319dde07c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj" Jul 29 15:27:47.381424 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.381340 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9qdl\" (UniqueName: \"kubernetes.io/projected/16cf7c54-77f6-45ea-91e4-327699f19c90-kube-api-access-b9qdl\") pod \"payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt\" (UID: \"16cf7c54-77f6-45ea-91e4-327699f19c90\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt" Jul 29 15:27:47.381424 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.381396 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf4qp\" (UniqueName: \"kubernetes.io/projected/6dbe1be3-6e15-44da-9d20-6d5319dde07c-kube-api-access-tf4qp\") pod \"payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj\" (UID: \"6dbe1be3-6e15-44da-9d20-6d5319dde07c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj" Jul 29 15:27:47.381535 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.381453 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16cf7c54-77f6-45ea-91e4-327699f19c90-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt\" (UID: \"16cf7c54-77f6-45ea-91e4-327699f19c90\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt" Jul 29 15:27:47.382045 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.382022 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6dbe1be3-6e15-44da-9d20-6d5319dde07c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj\" (UID: \"6dbe1be3-6e15-44da-9d20-6d5319dde07c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj" Jul 29 15:27:47.382151 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.382133 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16cf7c54-77f6-45ea-91e4-327699f19c90-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt\" (UID: \"16cf7c54-77f6-45ea-91e4-327699f19c90\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt" Jul 29 15:27:47.392922 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.392872 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf4qp\" (UniqueName: \"kubernetes.io/projected/6dbe1be3-6e15-44da-9d20-6d5319dde07c-kube-api-access-tf4qp\") pod \"payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj\" (UID: \"6dbe1be3-6e15-44da-9d20-6d5319dde07c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj" Jul 29 15:27:47.392922 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.392902 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9qdl\" (UniqueName: \"kubernetes.io/projected/16cf7c54-77f6-45ea-91e4-327699f19c90-kube-api-access-b9qdl\") pod \"payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt\" (UID: \"16cf7c54-77f6-45ea-91e4-327699f19c90\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt" Jul 29 15:27:47.451075 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.451053 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt" Jul 29 15:27:47.481846 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.481824 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj" Jul 29 15:27:47.803880 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.803857 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt"] Jul 29 15:27:47.804932 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:27:47.804909 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16cf7c54_77f6_45ea_91e4_327699f19c90.slice/crio-f7c1d184f1117683f3673062f7980787c48253face49db8be64add0633f49056 WatchSource:0}: Error finding container f7c1d184f1117683f3673062f7980787c48253face49db8be64add0633f49056: Status 404 returned error can't find the container with id f7c1d184f1117683f3673062f7980787c48253face49db8be64add0633f49056 Jul 29 15:27:47.834810 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:47.834790 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj"] Jul 29 15:27:47.835919 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:27:47.835895 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dbe1be3_6e15_44da_9d20_6d5319dde07c.slice/crio-d15cedd62dea35082019be49e4f251be44070abb52fb5998ca3c18fc2b7e689e WatchSource:0}: Error finding container d15cedd62dea35082019be49e4f251be44070abb52fb5998ca3c18fc2b7e689e: Status 404 returned error can't find the container with id d15cedd62dea35082019be49e4f251be44070abb52fb5998ca3c18fc2b7e689e Jul 29 15:27:48.638704 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:48.638672 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj" event={"ID":"6dbe1be3-6e15-44da-9d20-6d5319dde07c","Type":"ContainerStarted","Data":"8b70daff12d1329fb2f342139e3d139798a098437bc29f8b863fd66c1e54d7fb"} Jul 29 15:27:48.639141 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:48.638754 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj" Jul 29 15:27:48.639141 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:48.638779 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj" event={"ID":"6dbe1be3-6e15-44da-9d20-6d5319dde07c","Type":"ContainerStarted","Data":"d15cedd62dea35082019be49e4f251be44070abb52fb5998ca3c18fc2b7e689e"} Jul 29 15:27:48.640229 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:48.640207 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt" event={"ID":"16cf7c54-77f6-45ea-91e4-327699f19c90","Type":"ContainerStarted","Data":"0654bb2eca210dd9bdbee9e0d81cc99577fee61783996756ba0fb4f1dc76e8d1"} Jul 29 15:27:48.640332 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:48.640250 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt" event={"ID":"16cf7c54-77f6-45ea-91e4-327699f19c90","Type":"ContainerStarted","Data":"f7c1d184f1117683f3673062f7980787c48253face49db8be64add0633f49056"} Jul 29 15:27:48.640332 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:48.640270 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt" Jul 29 15:27:48.656213 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:48.656168 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj" podStartSLOduration=1.656154203 podStartE2EDuration="1.656154203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:27:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:27:48.65336328 +0000 UTC m=+3145.381512467" watchObservedRunningTime="2026-07-29 15:27:48.656154203 +0000 UTC m=+3145.384303431" Jul 29 15:27:48.673539 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:48.673496 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt" podStartSLOduration=1.6734837169999999 podStartE2EDuration="1.673483717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:27:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:27:48.670945778 +0000 UTC m=+3145.399094977" watchObservedRunningTime="2026-07-29 15:27:48.673483717 +0000 UTC m=+3145.401632904" Jul 29 15:27:53.974677 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:53.974650 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-fqmpx" Jul 29 15:27:53.975146 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:53.974697 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-fqmpx" Jul 29 15:27:53.994564 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:53.994536 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-fqmpx" Jul 29 15:27:54.697342 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:54.697308 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-fqmpx" Jul 29 15:27:54.730896 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:54.730865 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fqmpx"] Jul 29 15:27:56.686433 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:56.686395 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-fqmpx" podUID="62abd7b2-d785-4360-a439-ddf537627bd6" containerName="registry-server" containerID="cri-o://9089d05c4b4a9f5aeca8c8e5d3efc7a572a8f9836d934f403d2768ae80be3a1c" gracePeriod=2 Jul 29 15:27:56.940806 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:56.940754 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fqmpx" Jul 29 15:27:57.057611 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:57.057582 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cw968\" (UniqueName: \"kubernetes.io/projected/62abd7b2-d785-4360-a439-ddf537627bd6-kube-api-access-cw968\") pod \"62abd7b2-d785-4360-a439-ddf537627bd6\" (UID: \"62abd7b2-d785-4360-a439-ddf537627bd6\") " Jul 29 15:27:57.059875 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:57.059844 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62abd7b2-d785-4360-a439-ddf537627bd6-kube-api-access-cw968" (OuterVolumeSpecName: "kube-api-access-cw968") pod "62abd7b2-d785-4360-a439-ddf537627bd6" (UID: "62abd7b2-d785-4360-a439-ddf537627bd6"). InnerVolumeSpecName "kube-api-access-cw968". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:27:57.159145 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:57.159126 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cw968\" (UniqueName: \"kubernetes.io/projected/62abd7b2-d785-4360-a439-ddf537627bd6-kube-api-access-cw968\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:27:57.693050 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:57.693008 2580 generic.go:356] "Generic (PLEG): container finished" podID="62abd7b2-d785-4360-a439-ddf537627bd6" containerID="9089d05c4b4a9f5aeca8c8e5d3efc7a572a8f9836d934f403d2768ae80be3a1c" exitCode=0 Jul 29 15:27:57.693470 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:57.693065 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fqmpx" Jul 29 15:27:57.693470 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:57.693095 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fqmpx" event={"ID":"62abd7b2-d785-4360-a439-ddf537627bd6","Type":"ContainerDied","Data":"9089d05c4b4a9f5aeca8c8e5d3efc7a572a8f9836d934f403d2768ae80be3a1c"} Jul 29 15:27:57.693470 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:57.693142 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fqmpx" event={"ID":"62abd7b2-d785-4360-a439-ddf537627bd6","Type":"ContainerDied","Data":"d10a887d85a14d57ba50f36e364cd4cf0c9565070d833e0f3460a2a58d200ceb"} Jul 29 15:27:57.693470 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:57.693163 2580 scope.go:117] "RemoveContainer" containerID="9089d05c4b4a9f5aeca8c8e5d3efc7a572a8f9836d934f403d2768ae80be3a1c" Jul 29 15:27:57.703729 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:57.703704 2580 scope.go:117] "RemoveContainer" containerID="9089d05c4b4a9f5aeca8c8e5d3efc7a572a8f9836d934f403d2768ae80be3a1c" Jul 29 15:27:57.703969 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:27:57.703950 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9089d05c4b4a9f5aeca8c8e5d3efc7a572a8f9836d934f403d2768ae80be3a1c\": container with ID starting with 9089d05c4b4a9f5aeca8c8e5d3efc7a572a8f9836d934f403d2768ae80be3a1c not found: ID does not exist" containerID="9089d05c4b4a9f5aeca8c8e5d3efc7a572a8f9836d934f403d2768ae80be3a1c" Jul 29 15:27:57.704030 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:57.703981 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9089d05c4b4a9f5aeca8c8e5d3efc7a572a8f9836d934f403d2768ae80be3a1c"} err="failed to get container status \"9089d05c4b4a9f5aeca8c8e5d3efc7a572a8f9836d934f403d2768ae80be3a1c\": rpc error: code = NotFound desc = could not find container \"9089d05c4b4a9f5aeca8c8e5d3efc7a572a8f9836d934f403d2768ae80be3a1c\": container with ID starting with 9089d05c4b4a9f5aeca8c8e5d3efc7a572a8f9836d934f403d2768ae80be3a1c not found: ID does not exist" Jul 29 15:27:57.722416 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:57.722388 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fqmpx"] Jul 29 15:27:57.732375 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:57.732354 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fqmpx"] Jul 29 15:27:57.817959 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:57.817937 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62abd7b2-d785-4360-a439-ddf537627bd6" path="/var/lib/kubelet/pods/62abd7b2-d785-4360-a439-ddf537627bd6/volumes" Jul 29 15:27:59.648629 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:59.648598 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj" Jul 29 15:27:59.649049 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:27:59.648668 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt" Jul 29 15:28:11.331747 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:28:11.331688 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:28:18.246378 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:28:18.246329 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-45bsl_1695ee5c-3411-48e2-85e7-f11d527f6f5e/payload-processing/0.log" Jul 29 15:28:18.362606 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:28:18.362573 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj_6dbe1be3-6e15-44da-9d20-6d5319dde07c/payload-processing/0.log" Jul 29 15:28:27.141175 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:28:27.141141 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:28:32.993799 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:28:32.993759 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:29:08.420535 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:08.420496 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j_b8708b34-f25c-4494-a410-0d9c7d5ba1ee/payload-processing/0.log" Jul 29 15:29:08.545771 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:08.545741 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-45bsl_1695ee5c-3411-48e2-85e7-f11d527f6f5e/payload-processing/0.log" Jul 29 15:29:08.670905 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:08.670847 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj_6dbe1be3-6e15-44da-9d20-6d5319dde07c/payload-processing/0.log" Jul 29 15:29:08.949525 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:08.949463 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:29:09.272964 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:09.272921 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:29:09.533936 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:09.533853 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:29:12.049301 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:12.049212 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:29:12.150180 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:12.150134 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:29:12.233093 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:12.233065 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:29:12.570336 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:12.570300 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:29:12.730129 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:12.730081 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:29:12.825238 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:12.825146 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:29:12.829942 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:12.829921 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp"] Jul 29 15:29:12.830316 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:12.830304 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62abd7b2-d785-4360-a439-ddf537627bd6" containerName="registry-server" Jul 29 15:29:12.830357 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:12.830317 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="62abd7b2-d785-4360-a439-ddf537627bd6" containerName="registry-server" Jul 29 15:29:12.830401 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:12.830392 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="62abd7b2-d785-4360-a439-ddf537627bd6" containerName="registry-server" Jul 29 15:29:12.833465 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:12.833452 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp" Jul 29 15:29:12.835435 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:12.835419 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-04773a\"" Jul 29 15:29:12.835530 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:12.835422 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-04773a-dockercfg-5zggc\"" Jul 29 15:29:12.846149 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:12.846126 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp"] Jul 29 15:29:12.851192 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:12.851169 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt"] Jul 29 15:29:12.855236 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:12.855217 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt" Jul 29 15:29:12.868889 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:12.868867 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt"] Jul 29 15:29:12.947178 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:12.947145 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eca74a51-4a8a-4810-8097-99843809286d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt\" (UID: \"eca74a51-4a8a-4810-8097-99843809286d\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt" Jul 29 15:29:12.947383 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:12.947271 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stsxz\" (UniqueName: \"kubernetes.io/projected/eca74a51-4a8a-4810-8097-99843809286d-kube-api-access-stsxz\") pod \"payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt\" (UID: \"eca74a51-4a8a-4810-8097-99843809286d\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt" Jul 29 15:29:12.947383 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:12.947306 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhw54\" (UniqueName: \"kubernetes.io/projected/9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3-kube-api-access-nhw54\") pod \"payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp\" (UID: \"9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp" Jul 29 15:29:12.947383 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:12.947363 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp\" (UID: \"9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp" Jul 29 15:29:13.048563 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:13.048531 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp\" (UID: \"9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp" Jul 29 15:29:13.048708 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:13.048685 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eca74a51-4a8a-4810-8097-99843809286d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt\" (UID: \"eca74a51-4a8a-4810-8097-99843809286d\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt" Jul 29 15:29:13.048802 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:13.048763 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stsxz\" (UniqueName: \"kubernetes.io/projected/eca74a51-4a8a-4810-8097-99843809286d-kube-api-access-stsxz\") pod \"payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt\" (UID: \"eca74a51-4a8a-4810-8097-99843809286d\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt" Jul 29 15:29:13.048802 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:13.048793 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhw54\" (UniqueName: \"kubernetes.io/projected/9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3-kube-api-access-nhw54\") pod \"payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp\" (UID: \"9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp" Jul 29 15:29:13.049491 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:13.049465 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp\" (UID: \"9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp" Jul 29 15:29:13.049900 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:13.049519 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eca74a51-4a8a-4810-8097-99843809286d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt\" (UID: \"eca74a51-4a8a-4810-8097-99843809286d\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt" Jul 29 15:29:13.057030 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:13.056993 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stsxz\" (UniqueName: \"kubernetes.io/projected/eca74a51-4a8a-4810-8097-99843809286d-kube-api-access-stsxz\") pod \"payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt\" (UID: \"eca74a51-4a8a-4810-8097-99843809286d\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt" Jul 29 15:29:13.057309 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:13.057292 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhw54\" (UniqueName: \"kubernetes.io/projected/9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3-kube-api-access-nhw54\") pod \"payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp\" (UID: \"9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp" Jul 29 15:29:13.143527 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:13.143471 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp" Jul 29 15:29:13.165845 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:13.165818 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt" Jul 29 15:29:13.295359 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:13.295327 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp"] Jul 29 15:29:13.296888 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:29:13.296859 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d95c1ab_6cc2_4031_b6e2_cdd2eaa301d3.slice/crio-3c384101f9d7a37a9cd4d38f792e8d58a0b2ba4f254df5e42a755ac7da32ef49 WatchSource:0}: Error finding container 3c384101f9d7a37a9cd4d38f792e8d58a0b2ba4f254df5e42a755ac7da32ef49: Status 404 returned error can't find the container with id 3c384101f9d7a37a9cd4d38f792e8d58a0b2ba4f254df5e42a755ac7da32ef49 Jul 29 15:29:13.525696 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:13.525672 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt"] Jul 29 15:29:13.526344 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:29:13.526318 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeca74a51_4a8a_4810_8097_99843809286d.slice/crio-05a83541eb48eb165a2b6fee8873d999537fb064a4b7e26fab0814eaa46c580f WatchSource:0}: Error finding container 05a83541eb48eb165a2b6fee8873d999537fb064a4b7e26fab0814eaa46c580f: Status 404 returned error can't find the container with id 05a83541eb48eb165a2b6fee8873d999537fb064a4b7e26fab0814eaa46c580f Jul 29 15:29:14.125967 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:14.125935 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt" event={"ID":"eca74a51-4a8a-4810-8097-99843809286d","Type":"ContainerStarted","Data":"b70150e2ec0e2ab58138d2111dd5b972aa5e4efabbaa2d6d907ddf4c88d807fc"} Jul 29 15:29:14.126328 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:14.125997 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt" Jul 29 15:29:14.126328 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:14.126020 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt" event={"ID":"eca74a51-4a8a-4810-8097-99843809286d","Type":"ContainerStarted","Data":"05a83541eb48eb165a2b6fee8873d999537fb064a4b7e26fab0814eaa46c580f"} Jul 29 15:29:14.127447 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:14.127423 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp" event={"ID":"9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3","Type":"ContainerStarted","Data":"0699b3a73a28862386bb1013d8f3ce2aae813844bf6c5f14916ecbd6ff524ad8"} Jul 29 15:29:14.127532 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:14.127466 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp" event={"ID":"9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3","Type":"ContainerStarted","Data":"3c384101f9d7a37a9cd4d38f792e8d58a0b2ba4f254df5e42a755ac7da32ef49"} Jul 29 15:29:14.127532 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:14.127500 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp" Jul 29 15:29:14.143802 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:14.143747 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt" podStartSLOduration=2.143712116 podStartE2EDuration="2.143712116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:29:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:29:14.142426076 +0000 UTC m=+3230.870575265" watchObservedRunningTime="2026-07-29 15:29:14.143712116 +0000 UTC m=+3230.871861303" Jul 29 15:29:14.163009 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:14.162963 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp" podStartSLOduration=2.162946274 podStartE2EDuration="2.162946274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:29:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:29:14.160490411 +0000 UTC m=+3230.888639624" watchObservedRunningTime="2026-07-29 15:29:14.162946274 +0000 UTC m=+3230.891095464" Jul 29 15:29:25.135945 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:25.135899 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp" podUID="9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.88:9004: connect: connection refused" Jul 29 15:29:25.136390 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:25.136250 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt" Jul 29 15:29:35.134782 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:35.134754 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp" Jul 29 15:29:37.799014 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:37.798975 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-04773a-qtn4q"] Jul 29 15:29:37.803477 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:37.803452 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-04773a-qtn4q" Jul 29 15:29:37.807093 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:37.807076 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mmt6p\"" Jul 29 15:29:37.807259 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:37.807243 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 29 15:29:37.820091 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:37.820066 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-04773a-qtn4q"] Jul 29 15:29:37.829117 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:37.829098 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc0af3e5-7982-44d9-97ae-9ebd93ed2697-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-04773a-qtn4q\" (UID: \"fc0af3e5-7982-44d9-97ae-9ebd93ed2697\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-04773a-qtn4q" Jul 29 15:29:37.930574 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:37.930544 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc0af3e5-7982-44d9-97ae-9ebd93ed2697-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-04773a-qtn4q\" (UID: \"fc0af3e5-7982-44d9-97ae-9ebd93ed2697\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-04773a-qtn4q" Jul 29 15:29:37.931238 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:37.931201 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc0af3e5-7982-44d9-97ae-9ebd93ed2697-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-04773a-qtn4q\" (UID: \"fc0af3e5-7982-44d9-97ae-9ebd93ed2697\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-04773a-qtn4q" Jul 29 15:29:38.082840 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:38.082756 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:29:38.113800 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:38.113772 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-04773a-qtn4q" Jul 29 15:29:38.247258 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:38.247229 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-04773a-qtn4q"] Jul 29 15:29:38.249027 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:29:38.249002 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc0af3e5_7982_44d9_97ae_9ebd93ed2697.slice/crio-05ed7db093df9ffca98a5ad36a90b61d251f6eee953f3cdc36842f399549a05b WatchSource:0}: Error finding container 05ed7db093df9ffca98a5ad36a90b61d251f6eee953f3cdc36842f399549a05b: Status 404 returned error can't find the container with id 05ed7db093df9ffca98a5ad36a90b61d251f6eee953f3cdc36842f399549a05b Jul 29 15:29:38.264837 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:38.264811 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-04773a-qtn4q" event={"ID":"fc0af3e5-7982-44d9-97ae-9ebd93ed2697","Type":"ContainerStarted","Data":"05ed7db093df9ffca98a5ad36a90b61d251f6eee953f3cdc36842f399549a05b"} Jul 29 15:29:39.271716 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:39.271691 2580 generic.go:356] "Generic (PLEG): container finished" podID="fc0af3e5-7982-44d9-97ae-9ebd93ed2697" containerID="b1e266031756edb46c9c40bebd0b692df04d6365e73f2ac5407f03fa0cb4e099" exitCode=0 Jul 29 15:29:39.272052 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:39.271779 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-04773a-qtn4q" event={"ID":"fc0af3e5-7982-44d9-97ae-9ebd93ed2697","Type":"ContainerDied","Data":"b1e266031756edb46c9c40bebd0b692df04d6365e73f2ac5407f03fa0cb4e099"} Jul 29 15:29:40.414512 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:40.414487 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-04773a-qtn4q" Jul 29 15:29:40.451092 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:40.451065 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc0af3e5-7982-44d9-97ae-9ebd93ed2697-maas-api-service-ca\") pod \"fc0af3e5-7982-44d9-97ae-9ebd93ed2697\" (UID: \"fc0af3e5-7982-44d9-97ae-9ebd93ed2697\") " Jul 29 15:29:40.451408 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:40.451387 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc0af3e5-7982-44d9-97ae-9ebd93ed2697-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fc0af3e5-7982-44d9-97ae-9ebd93ed2697" (UID: "fc0af3e5-7982-44d9-97ae-9ebd93ed2697"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:29:40.551967 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:40.551898 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc0af3e5-7982-44d9-97ae-9ebd93ed2697-maas-api-service-ca\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:29:41.283585 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:41.283557 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-04773a-qtn4q" Jul 29 15:29:41.283819 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:41.283556 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-04773a-qtn4q" event={"ID":"fc0af3e5-7982-44d9-97ae-9ebd93ed2697","Type":"ContainerDied","Data":"05ed7db093df9ffca98a5ad36a90b61d251f6eee953f3cdc36842f399549a05b"} Jul 29 15:29:41.283819 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:41.283687 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="05ed7db093df9ffca98a5ad36a90b61d251f6eee953f3cdc36842f399549a05b" Jul 29 15:29:42.877361 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:42.877325 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:29:42.924328 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:42.924297 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt"] Jul 29 15:29:42.924583 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:42.924558 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt" podUID="eca74a51-4a8a-4810-8097-99843809286d" containerName="payload-processing" containerID="cri-o://b70150e2ec0e2ab58138d2111dd5b972aa5e4efabbaa2d6d907ddf4c88d807fc" gracePeriod=30 Jul 29 15:29:42.937200 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:42.937174 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp"] Jul 29 15:29:42.937415 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:42.937394 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp" podUID="9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3" containerName="payload-pre-processing" containerID="cri-o://0699b3a73a28862386bb1013d8f3ce2aae813844bf6c5f14916ecbd6ff524ad8" gracePeriod=30 Jul 29 15:29:42.961750 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:42.961711 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:29:43.033754 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.033717 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:29:43.093852 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.093781 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-04773a-qtn4q"] Jul 29 15:29:43.096461 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.096437 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-04773a-qtn4q"] Jul 29 15:29:43.109793 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.109529 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:29:43.228941 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.228921 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt" Jul 29 15:29:43.231888 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.231866 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp" Jul 29 15:29:43.273164 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.273136 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3-plugins-config-volume\") pod \"9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3\" (UID: \"9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3\") " Jul 29 15:29:43.273307 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.273201 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stsxz\" (UniqueName: \"kubernetes.io/projected/eca74a51-4a8a-4810-8097-99843809286d-kube-api-access-stsxz\") pod \"eca74a51-4a8a-4810-8097-99843809286d\" (UID: \"eca74a51-4a8a-4810-8097-99843809286d\") " Jul 29 15:29:43.273307 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.273282 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eca74a51-4a8a-4810-8097-99843809286d-plugins-config-volume\") pod \"eca74a51-4a8a-4810-8097-99843809286d\" (UID: \"eca74a51-4a8a-4810-8097-99843809286d\") " Jul 29 15:29:43.273443 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.273317 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhw54\" (UniqueName: \"kubernetes.io/projected/9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3-kube-api-access-nhw54\") pod \"9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3\" (UID: \"9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3\") " Jul 29 15:29:43.273530 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.273501 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3" (UID: "9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:29:43.273664 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.273635 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:29:43.273903 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.273871 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eca74a51-4a8a-4810-8097-99843809286d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eca74a51-4a8a-4810-8097-99843809286d" (UID: "eca74a51-4a8a-4810-8097-99843809286d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:29:43.275525 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.275494 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eca74a51-4a8a-4810-8097-99843809286d-kube-api-access-stsxz" (OuterVolumeSpecName: "kube-api-access-stsxz") pod "eca74a51-4a8a-4810-8097-99843809286d" (UID: "eca74a51-4a8a-4810-8097-99843809286d"). InnerVolumeSpecName "kube-api-access-stsxz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:29:43.275617 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.275580 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3-kube-api-access-nhw54" (OuterVolumeSpecName: "kube-api-access-nhw54") pod "9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3" (UID: "9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3"). InnerVolumeSpecName "kube-api-access-nhw54". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:29:43.295671 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.295647 2580 generic.go:356] "Generic (PLEG): container finished" podID="9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3" containerID="0699b3a73a28862386bb1013d8f3ce2aae813844bf6c5f14916ecbd6ff524ad8" exitCode=0 Jul 29 15:29:43.295797 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.295688 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp" Jul 29 15:29:43.295797 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.295751 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp" event={"ID":"9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3","Type":"ContainerDied","Data":"0699b3a73a28862386bb1013d8f3ce2aae813844bf6c5f14916ecbd6ff524ad8"} Jul 29 15:29:43.295899 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.295804 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp" event={"ID":"9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3","Type":"ContainerDied","Data":"3c384101f9d7a37a9cd4d38f792e8d58a0b2ba4f254df5e42a755ac7da32ef49"} Jul 29 15:29:43.295899 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.295831 2580 scope.go:117] "RemoveContainer" containerID="0699b3a73a28862386bb1013d8f3ce2aae813844bf6c5f14916ecbd6ff524ad8" Jul 29 15:29:43.297085 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.297064 2580 generic.go:356] "Generic (PLEG): container finished" podID="eca74a51-4a8a-4810-8097-99843809286d" containerID="b70150e2ec0e2ab58138d2111dd5b972aa5e4efabbaa2d6d907ddf4c88d807fc" exitCode=0 Jul 29 15:29:43.297181 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.297107 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt" Jul 29 15:29:43.297181 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.297160 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt" event={"ID":"eca74a51-4a8a-4810-8097-99843809286d","Type":"ContainerDied","Data":"b70150e2ec0e2ab58138d2111dd5b972aa5e4efabbaa2d6d907ddf4c88d807fc"} Jul 29 15:29:43.297283 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.297205 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt" event={"ID":"eca74a51-4a8a-4810-8097-99843809286d","Type":"ContainerDied","Data":"05a83541eb48eb165a2b6fee8873d999537fb064a4b7e26fab0814eaa46c580f"} Jul 29 15:29:43.309123 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.309106 2580 scope.go:117] "RemoveContainer" containerID="0699b3a73a28862386bb1013d8f3ce2aae813844bf6c5f14916ecbd6ff524ad8" Jul 29 15:29:43.309576 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:29:43.309559 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0699b3a73a28862386bb1013d8f3ce2aae813844bf6c5f14916ecbd6ff524ad8\": container with ID starting with 0699b3a73a28862386bb1013d8f3ce2aae813844bf6c5f14916ecbd6ff524ad8 not found: ID does not exist" containerID="0699b3a73a28862386bb1013d8f3ce2aae813844bf6c5f14916ecbd6ff524ad8" Jul 29 15:29:43.309656 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.309587 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0699b3a73a28862386bb1013d8f3ce2aae813844bf6c5f14916ecbd6ff524ad8"} err="failed to get container status \"0699b3a73a28862386bb1013d8f3ce2aae813844bf6c5f14916ecbd6ff524ad8\": rpc error: code = NotFound desc = could not find container \"0699b3a73a28862386bb1013d8f3ce2aae813844bf6c5f14916ecbd6ff524ad8\": container with ID starting with 0699b3a73a28862386bb1013d8f3ce2aae813844bf6c5f14916ecbd6ff524ad8 not found: ID does not exist" Jul 29 15:29:43.309656 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.309604 2580 scope.go:117] "RemoveContainer" containerID="b70150e2ec0e2ab58138d2111dd5b972aa5e4efabbaa2d6d907ddf4c88d807fc" Jul 29 15:29:43.318052 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.318036 2580 scope.go:117] "RemoveContainer" containerID="b70150e2ec0e2ab58138d2111dd5b972aa5e4efabbaa2d6d907ddf4c88d807fc" Jul 29 15:29:43.318322 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:29:43.318301 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b70150e2ec0e2ab58138d2111dd5b972aa5e4efabbaa2d6d907ddf4c88d807fc\": container with ID starting with b70150e2ec0e2ab58138d2111dd5b972aa5e4efabbaa2d6d907ddf4c88d807fc not found: ID does not exist" containerID="b70150e2ec0e2ab58138d2111dd5b972aa5e4efabbaa2d6d907ddf4c88d807fc" Jul 29 15:29:43.318374 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.318329 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b70150e2ec0e2ab58138d2111dd5b972aa5e4efabbaa2d6d907ddf4c88d807fc"} err="failed to get container status \"b70150e2ec0e2ab58138d2111dd5b972aa5e4efabbaa2d6d907ddf4c88d807fc\": rpc error: code = NotFound desc = could not find container \"b70150e2ec0e2ab58138d2111dd5b972aa5e4efabbaa2d6d907ddf4c88d807fc\": container with ID starting with b70150e2ec0e2ab58138d2111dd5b972aa5e4efabbaa2d6d907ddf4c88d807fc not found: ID does not exist" Jul 29 15:29:43.319280 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.319259 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp"] Jul 29 15:29:43.323297 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.323279 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-04773a-798859bd68-rqvmp"] Jul 29 15:29:43.333132 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.333112 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt"] Jul 29 15:29:43.337026 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.337004 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-04773a-74597894f6-747gt"] Jul 29 15:29:43.374307 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.374288 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stsxz\" (UniqueName: \"kubernetes.io/projected/eca74a51-4a8a-4810-8097-99843809286d-kube-api-access-stsxz\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:29:43.374394 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.374309 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eca74a51-4a8a-4810-8097-99843809286d-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:29:43.374394 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.374319 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhw54\" (UniqueName: \"kubernetes.io/projected/9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3-kube-api-access-nhw54\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:29:43.471539 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.471470 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:29:43.818508 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.818481 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3" path="/var/lib/kubelet/pods/9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3/volumes" Jul 29 15:29:43.818847 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.818833 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eca74a51-4a8a-4810-8097-99843809286d" path="/var/lib/kubelet/pods/eca74a51-4a8a-4810-8097-99843809286d/volumes" Jul 29 15:29:43.819119 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:43.819107 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc0af3e5-7982-44d9-97ae-9ebd93ed2697" path="/var/lib/kubelet/pods/fc0af3e5-7982-44d9-97ae-9ebd93ed2697/volumes" Jul 29 15:29:49.320937 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:49.320903 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:29:49.421689 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:49.421645 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:29:51.007569 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:51.007532 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:29:51.108642 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:51.108610 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:29:53.256766 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:53.256736 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3261e1-7q89z"] Jul 29 15:29:53.257330 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:53.257310 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3" containerName="payload-pre-processing" Jul 29 15:29:53.257426 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:53.257333 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3" containerName="payload-pre-processing" Jul 29 15:29:53.257426 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:53.257382 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eca74a51-4a8a-4810-8097-99843809286d" containerName="payload-processing" Jul 29 15:29:53.257426 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:53.257391 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="eca74a51-4a8a-4810-8097-99843809286d" containerName="payload-processing" Jul 29 15:29:53.257426 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:53.257405 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc0af3e5-7982-44d9-97ae-9ebd93ed2697" containerName="revoke-keys" Jul 29 15:29:53.257426 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:53.257414 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc0af3e5-7982-44d9-97ae-9ebd93ed2697" containerName="revoke-keys" Jul 29 15:29:53.257672 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:53.257512 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="eca74a51-4a8a-4810-8097-99843809286d" containerName="payload-processing" Jul 29 15:29:53.257672 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:53.257526 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d95c1ab-6cc2-4031-b6e2-cdd2eaa301d3" containerName="payload-pre-processing" Jul 29 15:29:53.257672 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:53.257536 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc0af3e5-7982-44d9-97ae-9ebd93ed2697" containerName="revoke-keys" Jul 29 15:29:53.262294 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:53.262273 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3261e1-7q89z" Jul 29 15:29:53.264680 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:53.264662 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 29 15:29:53.264968 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:53.264949 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mmt6p\"" Jul 29 15:29:53.270320 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:53.270167 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3261e1-7q89z"] Jul 29 15:29:53.368171 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:53.368132 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9abdc318-4bc8-439d-bb56-a0acb015d605-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-3261e1-7q89z\" (UID: \"9abdc318-4bc8-439d-bb56-a0acb015d605\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3261e1-7q89z" Jul 29 15:29:53.469403 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:53.469372 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9abdc318-4bc8-439d-bb56-a0acb015d605-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-3261e1-7q89z\" (UID: \"9abdc318-4bc8-439d-bb56-a0acb015d605\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3261e1-7q89z" Jul 29 15:29:53.469990 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:53.469970 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9abdc318-4bc8-439d-bb56-a0acb015d605-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-3261e1-7q89z\" (UID: \"9abdc318-4bc8-439d-bb56-a0acb015d605\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3261e1-7q89z" Jul 29 15:29:53.574505 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:53.574415 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3261e1-7q89z" Jul 29 15:29:53.916340 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:53.916311 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3261e1-7q89z"] Jul 29 15:29:53.917757 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:29:53.917705 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9abdc318_4bc8_439d_bb56_a0acb015d605.slice/crio-125f3abfb5420501366768864e8a9407b7270f0a32fcea912b111c9f6aff0451 WatchSource:0}: Error finding container 125f3abfb5420501366768864e8a9407b7270f0a32fcea912b111c9f6aff0451: Status 404 returned error can't find the container with id 125f3abfb5420501366768864e8a9407b7270f0a32fcea912b111c9f6aff0451 Jul 29 15:29:54.362651 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:54.362618 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3261e1-7q89z" event={"ID":"9abdc318-4bc8-439d-bb56-a0acb015d605","Type":"ContainerStarted","Data":"3e86359c42c6f0d313bc25ab80d1d08983ff1b364208722ae7eae38033e81ada"} Jul 29 15:29:54.363035 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:54.362672 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3261e1-7q89z" event={"ID":"9abdc318-4bc8-439d-bb56-a0acb015d605","Type":"ContainerStarted","Data":"125f3abfb5420501366768864e8a9407b7270f0a32fcea912b111c9f6aff0451"} Jul 29 15:29:54.383911 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:54.383864 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3261e1-7q89z" podStartSLOduration=1.383850731 podStartE2EDuration="1.383850731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:29:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:29:54.381715365 +0000 UTC m=+3271.109864554" watchObservedRunningTime="2026-07-29 15:29:54.383850731 +0000 UTC m=+3271.111999959" Jul 29 15:29:55.369477 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:55.369441 2580 generic.go:356] "Generic (PLEG): container finished" podID="9abdc318-4bc8-439d-bb56-a0acb015d605" containerID="3e86359c42c6f0d313bc25ab80d1d08983ff1b364208722ae7eae38033e81ada" exitCode=0 Jul 29 15:29:55.369869 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:55.369527 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3261e1-7q89z" event={"ID":"9abdc318-4bc8-439d-bb56-a0acb015d605","Type":"ContainerDied","Data":"3e86359c42c6f0d313bc25ab80d1d08983ff1b364208722ae7eae38033e81ada"} Jul 29 15:29:56.514436 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:56.514415 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3261e1-7q89z" Jul 29 15:29:56.599999 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:56.599973 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9abdc318-4bc8-439d-bb56-a0acb015d605-maas-api-service-ca\") pod \"9abdc318-4bc8-439d-bb56-a0acb015d605\" (UID: \"9abdc318-4bc8-439d-bb56-a0acb015d605\") " Jul 29 15:29:56.600312 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:56.600291 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9abdc318-4bc8-439d-bb56-a0acb015d605-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9abdc318-4bc8-439d-bb56-a0acb015d605" (UID: "9abdc318-4bc8-439d-bb56-a0acb015d605"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:29:56.701171 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:56.701097 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9abdc318-4bc8-439d-bb56-a0acb015d605-maas-api-service-ca\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:29:57.382335 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:57.382296 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3261e1-7q89z" event={"ID":"9abdc318-4bc8-439d-bb56-a0acb015d605","Type":"ContainerDied","Data":"125f3abfb5420501366768864e8a9407b7270f0a32fcea912b111c9f6aff0451"} Jul 29 15:29:57.382508 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:57.382345 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="125f3abfb5420501366768864e8a9407b7270f0a32fcea912b111c9f6aff0451" Jul 29 15:29:57.382508 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:57.382307 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3261e1-7q89z" Jul 29 15:29:59.887068 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:29:59.887010 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:30:00.144173 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.144080 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755650-654vm"] Jul 29 15:30:00.144609 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.144581 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9abdc318-4bc8-439d-bb56-a0acb015d605" containerName="revoke-keys" Jul 29 15:30:00.144609 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.144603 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9abdc318-4bc8-439d-bb56-a0acb015d605" containerName="revoke-keys" Jul 29 15:30:00.144852 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.144707 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9abdc318-4bc8-439d-bb56-a0acb015d605" containerName="revoke-keys" Jul 29 15:30:00.148108 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.148085 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c73214-29755650-csc2r"] Jul 29 15:30:00.148247 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.148194 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755650-654vm" Jul 29 15:30:00.150992 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.150971 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mmt6p\"" Jul 29 15:30:00.152707 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.152687 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-93e799-29755650-hdp7b"] Jul 29 15:30:00.152822 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.152804 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c73214-29755650-csc2r" Jul 29 15:30:00.159890 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.159871 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a4092e-29755650-9h9cj"] Jul 29 15:30:00.160013 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.159993 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-93e799-29755650-hdp7b" Jul 29 15:30:00.165733 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.165686 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3261e1-29755650-vb9kk"] Jul 29 15:30:00.165842 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.165796 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a4092e-29755650-9h9cj" Jul 29 15:30:00.167557 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.167532 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755650-654vm" Jul 29 15:30:00.170332 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.170312 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755650-654vm"] Jul 29 15:30:00.170422 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.170350 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c73214-29755650-csc2r"] Jul 29 15:30:00.170422 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.170373 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-93e799-29755650-hdp7b"] Jul 29 15:30:00.170422 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.170394 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a4092e-29755650-9h9cj"] Jul 29 15:30:00.170546 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.170430 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3261e1-29755650-vb9kk" Jul 29 15:30:00.176776 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.176757 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3261e1-29755650-vb9kk"] Jul 29 15:30:00.176919 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.176902 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c73214-29755650-csc2r" Jul 29 15:30:00.221451 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.221429 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-93e799-29755650-hdp7b" Jul 29 15:30:00.236305 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.235871 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a4092e-29755650-9h9cj" Jul 29 15:30:00.283282 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.283107 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3261e1-29755650-vb9kk" Jul 29 15:30:00.583408 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.583379 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755650-654vm"] Jul 29 15:30:00.583902 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:30:00.583878 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93fea96f_35f0_4de2_8d5d_641566ff956f.slice/crio-c51f155f148082b9aaa53a546b03ac2456aae16218183663147f09e98116c61c WatchSource:0}: Error finding container c51f155f148082b9aaa53a546b03ac2456aae16218183663147f09e98116c61c: Status 404 returned error can't find the container with id c51f155f148082b9aaa53a546b03ac2456aae16218183663147f09e98116c61c Jul 29 15:30:00.626289 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.626267 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c73214-29755650-csc2r"] Jul 29 15:30:00.627980 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:30:00.627954 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf51fa978_0c3d_4ff8_a827_91b044de33bf.slice/crio-ae4b9adb1591915b30a24a245b869093ef5e66ad323c831b88d66cef38cf9bd1 WatchSource:0}: Error finding container ae4b9adb1591915b30a24a245b869093ef5e66ad323c831b88d66cef38cf9bd1: Status 404 returned error can't find the container with id ae4b9adb1591915b30a24a245b869093ef5e66ad323c831b88d66cef38cf9bd1 Jul 29 15:30:00.657735 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.657697 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-93e799-29755650-hdp7b"] Jul 29 15:30:00.660285 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:30:00.660185 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52031ac1_81e4_43d6_aa79_762f1c461b8f.slice/crio-168f3c3cad35a07cd4abdc6927dcb2f9889c9fae2d9153727361cf0d703135fc WatchSource:0}: Error finding container 168f3c3cad35a07cd4abdc6927dcb2f9889c9fae2d9153727361cf0d703135fc: Status 404 returned error can't find the container with id 168f3c3cad35a07cd4abdc6927dcb2f9889c9fae2d9153727361cf0d703135fc Jul 29 15:30:00.667788 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.667708 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a4092e-29755650-9h9cj"] Jul 29 15:30:00.668703 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:30:00.668684 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73801393_74d6_40bb_998b_faa7589273a0.slice/crio-9c9d9313128fdb6fb2137752443d6ed74e37a916c0994585457047c2e1f7bcb2 WatchSource:0}: Error finding container 9c9d9313128fdb6fb2137752443d6ed74e37a916c0994585457047c2e1f7bcb2: Status 404 returned error can't find the container with id 9c9d9313128fdb6fb2137752443d6ed74e37a916c0994585457047c2e1f7bcb2 Jul 29 15:30:00.681390 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:00.681363 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3261e1-29755650-vb9kk"] Jul 29 15:30:00.691939 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:30:00.691907 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b5421c3_ac30_4f0e_8b91_82667e94eb25.slice/crio-2ce6609aed756237371140a534e437320fc4f49b426371f7a8af92749cd1b5f8 WatchSource:0}: Error finding container 2ce6609aed756237371140a534e437320fc4f49b426371f7a8af92749cd1b5f8: Status 404 returned error can't find the container with id 2ce6609aed756237371140a534e437320fc4f49b426371f7a8af92749cd1b5f8 Jul 29 15:30:01.416146 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:01.416110 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-93e799-29755650-hdp7b" event={"ID":"52031ac1-81e4-43d6-aa79-762f1c461b8f","Type":"ContainerStarted","Data":"2e7485d1085233b1164c3587d6e4f726dbe26caf113c98e3941d40ddf1416f83"} Jul 29 15:30:01.416568 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:01.416184 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-93e799-29755650-hdp7b" event={"ID":"52031ac1-81e4-43d6-aa79-762f1c461b8f","Type":"ContainerStarted","Data":"168f3c3cad35a07cd4abdc6927dcb2f9889c9fae2d9153727361cf0d703135fc"} Jul 29 15:30:01.417882 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:01.417853 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755650-654vm" event={"ID":"93fea96f-35f0-4de2-8d5d-641566ff956f","Type":"ContainerStarted","Data":"2e10c1666b01dfaccb643bc14791cdec69c5fcaf9ee9698495b50146d9307816"} Jul 29 15:30:01.418013 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:01.417907 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755650-654vm" event={"ID":"93fea96f-35f0-4de2-8d5d-641566ff956f","Type":"ContainerStarted","Data":"c51f155f148082b9aaa53a546b03ac2456aae16218183663147f09e98116c61c"} Jul 29 15:30:01.419350 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:01.419327 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3261e1-29755650-vb9kk" event={"ID":"2b5421c3-ac30-4f0e-8b91-82667e94eb25","Type":"ContainerStarted","Data":"b0f8aa59014062ffc973fb5935951a26c6e006e67d62de540bb8c254f9566747"} Jul 29 15:30:01.419651 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:01.419373 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3261e1-29755650-vb9kk" event={"ID":"2b5421c3-ac30-4f0e-8b91-82667e94eb25","Type":"ContainerStarted","Data":"2ce6609aed756237371140a534e437320fc4f49b426371f7a8af92749cd1b5f8"} Jul 29 15:30:01.420887 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:01.420864 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c73214-29755650-csc2r" event={"ID":"f51fa978-0c3d-4ff8-a827-91b044de33bf","Type":"ContainerStarted","Data":"b444666825ae8d10344e260acf5fa17372f889f1e8b6734a4b536c47349cbfe8"} Jul 29 15:30:01.420987 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:01.420902 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c73214-29755650-csc2r" event={"ID":"f51fa978-0c3d-4ff8-a827-91b044de33bf","Type":"ContainerStarted","Data":"ae4b9adb1591915b30a24a245b869093ef5e66ad323c831b88d66cef38cf9bd1"} Jul 29 15:30:01.422201 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:01.422180 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a4092e-29755650-9h9cj" event={"ID":"73801393-74d6-40bb-998b-faa7589273a0","Type":"ContainerStarted","Data":"ce567285295f592fb07c0b11408d31b15b73bc7dcb252b26372151a9de4e664c"} Jul 29 15:30:01.422281 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:01.422223 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a4092e-29755650-9h9cj" event={"ID":"73801393-74d6-40bb-998b-faa7589273a0","Type":"ContainerStarted","Data":"9c9d9313128fdb6fb2137752443d6ed74e37a916c0994585457047c2e1f7bcb2"} Jul 29 15:30:01.433003 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:01.432958 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-93e799-29755650-hdp7b" podStartSLOduration=1.432946031 podStartE2EDuration="1.432946031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:30:01.429864361 +0000 UTC m=+3278.158013550" watchObservedRunningTime="2026-07-29 15:30:01.432946031 +0000 UTC m=+3278.161095253" Jul 29 15:30:01.442554 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:01.442515 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c73214-29755650-csc2r" podStartSLOduration=1.442506248 podStartE2EDuration="1.442506248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:30:01.441502416 +0000 UTC m=+3278.169651617" watchObservedRunningTime="2026-07-29 15:30:01.442506248 +0000 UTC m=+3278.170655437" Jul 29 15:30:01.453851 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:01.453808 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a4092e-29755650-9h9cj" podStartSLOduration=1.453791007 podStartE2EDuration="1.453791007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:30:01.452746612 +0000 UTC m=+3278.180895826" watchObservedRunningTime="2026-07-29 15:30:01.453791007 +0000 UTC m=+3278.181940197" Jul 29 15:30:01.465791 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:01.465754 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755650-654vm" podStartSLOduration=1.465742515 podStartE2EDuration="1.465742515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:30:01.464081943 +0000 UTC m=+3278.192231153" watchObservedRunningTime="2026-07-29 15:30:01.465742515 +0000 UTC m=+3278.193891705" Jul 29 15:30:01.479025 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:01.478992 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3261e1-29755650-vb9kk" podStartSLOduration=1.478981822 podStartE2EDuration="1.478981822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:30:01.47697666 +0000 UTC m=+3278.205125849" watchObservedRunningTime="2026-07-29 15:30:01.478981822 +0000 UTC m=+3278.207131010" Jul 29 15:30:02.430171 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:02.430140 2580 generic.go:356] "Generic (PLEG): container finished" podID="f51fa978-0c3d-4ff8-a827-91b044de33bf" containerID="b444666825ae8d10344e260acf5fa17372f889f1e8b6734a4b536c47349cbfe8" exitCode=0 Jul 29 15:30:02.430532 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:02.430231 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c73214-29755650-csc2r" event={"ID":"f51fa978-0c3d-4ff8-a827-91b044de33bf","Type":"ContainerDied","Data":"b444666825ae8d10344e260acf5fa17372f889f1e8b6734a4b536c47349cbfe8"} Jul 29 15:30:02.431762 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:02.431738 2580 generic.go:356] "Generic (PLEG): container finished" podID="73801393-74d6-40bb-998b-faa7589273a0" containerID="ce567285295f592fb07c0b11408d31b15b73bc7dcb252b26372151a9de4e664c" exitCode=0 Jul 29 15:30:02.431849 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:02.431816 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a4092e-29755650-9h9cj" event={"ID":"73801393-74d6-40bb-998b-faa7589273a0","Type":"ContainerDied","Data":"ce567285295f592fb07c0b11408d31b15b73bc7dcb252b26372151a9de4e664c"} Jul 29 15:30:02.433338 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:02.433316 2580 generic.go:356] "Generic (PLEG): container finished" podID="52031ac1-81e4-43d6-aa79-762f1c461b8f" containerID="2e7485d1085233b1164c3587d6e4f726dbe26caf113c98e3941d40ddf1416f83" exitCode=0 Jul 29 15:30:02.433436 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:02.433391 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-93e799-29755650-hdp7b" event={"ID":"52031ac1-81e4-43d6-aa79-762f1c461b8f","Type":"ContainerDied","Data":"2e7485d1085233b1164c3587d6e4f726dbe26caf113c98e3941d40ddf1416f83"} Jul 29 15:30:02.434868 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:02.434851 2580 generic.go:356] "Generic (PLEG): container finished" podID="93fea96f-35f0-4de2-8d5d-641566ff956f" containerID="2e10c1666b01dfaccb643bc14791cdec69c5fcaf9ee9698495b50146d9307816" exitCode=0 Jul 29 15:30:02.434951 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:02.434873 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755650-654vm" event={"ID":"93fea96f-35f0-4de2-8d5d-641566ff956f","Type":"ContainerDied","Data":"2e10c1666b01dfaccb643bc14791cdec69c5fcaf9ee9698495b50146d9307816"} Jul 29 15:30:02.436308 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:02.436265 2580 generic.go:356] "Generic (PLEG): container finished" podID="2b5421c3-ac30-4f0e-8b91-82667e94eb25" containerID="b0f8aa59014062ffc973fb5935951a26c6e006e67d62de540bb8c254f9566747" exitCode=0 Jul 29 15:30:02.436308 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:02.436287 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3261e1-29755650-vb9kk" event={"ID":"2b5421c3-ac30-4f0e-8b91-82667e94eb25","Type":"ContainerDied","Data":"b0f8aa59014062ffc973fb5935951a26c6e006e67d62de540bb8c254f9566747"} Jul 29 15:30:03.586840 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:03.586818 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a4092e-29755650-9h9cj" Jul 29 15:30:03.650959 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:03.650928 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-93e799-29755650-hdp7b" Jul 29 15:30:03.674504 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:03.674481 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3261e1-29755650-vb9kk" Jul 29 15:30:03.681386 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:03.681364 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755650-654vm" Jul 29 15:30:03.684900 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:03.684882 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c73214-29755650-csc2r" Jul 29 15:30:04.450092 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:04.450064 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755650-654vm" Jul 29 15:30:04.450092 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:04.450070 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755650-654vm" event={"ID":"93fea96f-35f0-4de2-8d5d-641566ff956f","Type":"ContainerDied","Data":"c51f155f148082b9aaa53a546b03ac2456aae16218183663147f09e98116c61c"} Jul 29 15:30:04.450314 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:04.450144 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c51f155f148082b9aaa53a546b03ac2456aae16218183663147f09e98116c61c" Jul 29 15:30:04.451488 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:04.451466 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3261e1-29755650-vb9kk" Jul 29 15:30:04.451488 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:04.451474 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3261e1-29755650-vb9kk" event={"ID":"2b5421c3-ac30-4f0e-8b91-82667e94eb25","Type":"ContainerDied","Data":"2ce6609aed756237371140a534e437320fc4f49b426371f7a8af92749cd1b5f8"} Jul 29 15:30:04.451705 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:04.451542 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ce6609aed756237371140a534e437320fc4f49b426371f7a8af92749cd1b5f8" Jul 29 15:30:04.453051 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:04.453032 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c73214-29755650-csc2r" Jul 29 15:30:04.453182 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:04.453066 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c73214-29755650-csc2r" event={"ID":"f51fa978-0c3d-4ff8-a827-91b044de33bf","Type":"ContainerDied","Data":"ae4b9adb1591915b30a24a245b869093ef5e66ad323c831b88d66cef38cf9bd1"} Jul 29 15:30:04.453182 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:04.453101 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae4b9adb1591915b30a24a245b869093ef5e66ad323c831b88d66cef38cf9bd1" Jul 29 15:30:04.454989 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:04.454960 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a4092e-29755650-9h9cj" Jul 29 15:30:04.455134 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:04.454995 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a4092e-29755650-9h9cj" event={"ID":"73801393-74d6-40bb-998b-faa7589273a0","Type":"ContainerDied","Data":"9c9d9313128fdb6fb2137752443d6ed74e37a916c0994585457047c2e1f7bcb2"} Jul 29 15:30:04.455134 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:04.455031 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c9d9313128fdb6fb2137752443d6ed74e37a916c0994585457047c2e1f7bcb2" Jul 29 15:30:04.456509 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:04.456488 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-93e799-29755650-hdp7b" event={"ID":"52031ac1-81e4-43d6-aa79-762f1c461b8f","Type":"ContainerDied","Data":"168f3c3cad35a07cd4abdc6927dcb2f9889c9fae2d9153727361cf0d703135fc"} Jul 29 15:30:04.456509 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:04.456507 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-93e799-29755650-hdp7b" Jul 29 15:30:04.456666 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:04.456526 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="168f3c3cad35a07cd4abdc6927dcb2f9889c9fae2d9153727361cf0d703135fc" Jul 29 15:30:08.786603 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:08.786574 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:30:08.788537 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:08.788480 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3261e1-29755650-vb9kk"] Jul 29 15:30:08.791325 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:08.791301 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-3261e1-29755650-vb9kk"] Jul 29 15:30:08.823491 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:08.823470 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j"] Jul 29 15:30:08.823676 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:08.823653 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j" podUID="b8708b34-f25c-4494-a410-0d9c7d5ba1ee" containerName="payload-processing" containerID="cri-o://9cb3473cad4ca081d7fcd7b842f6ff260e855cb5a9983963588384c92210b999" gracePeriod=30 Jul 29 15:30:08.833083 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:08.833060 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw"] Jul 29 15:30:08.833495 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:08.833444 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw" podUID="0acec3d3-3ce1-4086-bec0-08be2669f967" containerName="payload-pre-processing" containerID="cri-o://8d0bf9261a41a5f007871941aeca977dc61f423d2ea6220f58e5bbeb6c4ae22a" gracePeriod=30 Jul 29 15:30:08.980230 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:08.980190 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3261e1-7q89z"] Jul 29 15:30:08.986477 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:08.986424 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-3261e1-7q89z"] Jul 29 15:30:09.033281 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.033252 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:30:09.106947 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.106913 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:30:09.121249 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.121227 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j" Jul 29 15:30:09.124078 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.124061 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw" Jul 29 15:30:09.208841 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.208811 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8708b34-f25c-4494-a410-0d9c7d5ba1ee-plugins-config-volume\") pod \"b8708b34-f25c-4494-a410-0d9c7d5ba1ee\" (UID: \"b8708b34-f25c-4494-a410-0d9c7d5ba1ee\") " Jul 29 15:30:09.208973 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.208863 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cv95g\" (UniqueName: \"kubernetes.io/projected/0acec3d3-3ce1-4086-bec0-08be2669f967-kube-api-access-cv95g\") pod \"0acec3d3-3ce1-4086-bec0-08be2669f967\" (UID: \"0acec3d3-3ce1-4086-bec0-08be2669f967\") " Jul 29 15:30:09.208973 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.208921 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0acec3d3-3ce1-4086-bec0-08be2669f967-plugins-config-volume\") pod \"0acec3d3-3ce1-4086-bec0-08be2669f967\" (UID: \"0acec3d3-3ce1-4086-bec0-08be2669f967\") " Jul 29 15:30:09.209061 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.208987 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjzv5\" (UniqueName: \"kubernetes.io/projected/b8708b34-f25c-4494-a410-0d9c7d5ba1ee-kube-api-access-bjzv5\") pod \"b8708b34-f25c-4494-a410-0d9c7d5ba1ee\" (UID: \"b8708b34-f25c-4494-a410-0d9c7d5ba1ee\") " Jul 29 15:30:09.209267 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.209238 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8708b34-f25c-4494-a410-0d9c7d5ba1ee-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b8708b34-f25c-4494-a410-0d9c7d5ba1ee" (UID: "b8708b34-f25c-4494-a410-0d9c7d5ba1ee"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:30:09.209360 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.209309 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0acec3d3-3ce1-4086-bec0-08be2669f967-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0acec3d3-3ce1-4086-bec0-08be2669f967" (UID: "0acec3d3-3ce1-4086-bec0-08be2669f967"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:30:09.210995 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.210968 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8708b34-f25c-4494-a410-0d9c7d5ba1ee-kube-api-access-bjzv5" (OuterVolumeSpecName: "kube-api-access-bjzv5") pod "b8708b34-f25c-4494-a410-0d9c7d5ba1ee" (UID: "b8708b34-f25c-4494-a410-0d9c7d5ba1ee"). InnerVolumeSpecName "kube-api-access-bjzv5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:30:09.211088 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.211006 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0acec3d3-3ce1-4086-bec0-08be2669f967-kube-api-access-cv95g" (OuterVolumeSpecName: "kube-api-access-cv95g") pod "0acec3d3-3ce1-4086-bec0-08be2669f967" (UID: "0acec3d3-3ce1-4086-bec0-08be2669f967"). InnerVolumeSpecName "kube-api-access-cv95g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:30:09.310681 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.310607 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8708b34-f25c-4494-a410-0d9c7d5ba1ee-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:30:09.310681 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.310635 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cv95g\" (UniqueName: \"kubernetes.io/projected/0acec3d3-3ce1-4086-bec0-08be2669f967-kube-api-access-cv95g\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:30:09.310681 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.310647 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0acec3d3-3ce1-4086-bec0-08be2669f967-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:30:09.310681 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.310660 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjzv5\" (UniqueName: \"kubernetes.io/projected/b8708b34-f25c-4494-a410-0d9c7d5ba1ee-kube-api-access-bjzv5\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:30:09.417954 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.417920 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:30:09.489034 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.489000 2580 generic.go:356] "Generic (PLEG): container finished" podID="0acec3d3-3ce1-4086-bec0-08be2669f967" containerID="8d0bf9261a41a5f007871941aeca977dc61f423d2ea6220f58e5bbeb6c4ae22a" exitCode=0 Jul 29 15:30:09.489216 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.489049 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw" Jul 29 15:30:09.489216 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.489089 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw" event={"ID":"0acec3d3-3ce1-4086-bec0-08be2669f967","Type":"ContainerDied","Data":"8d0bf9261a41a5f007871941aeca977dc61f423d2ea6220f58e5bbeb6c4ae22a"} Jul 29 15:30:09.489216 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.489185 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw" event={"ID":"0acec3d3-3ce1-4086-bec0-08be2669f967","Type":"ContainerDied","Data":"2a6fd1b0c1e68eb486d3ff7c11056bef4d8926833c046f868cd624e3416f0a3c"} Jul 29 15:30:09.489216 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.489215 2580 scope.go:117] "RemoveContainer" containerID="8d0bf9261a41a5f007871941aeca977dc61f423d2ea6220f58e5bbeb6c4ae22a" Jul 29 15:30:09.490760 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.490709 2580 generic.go:356] "Generic (PLEG): container finished" podID="b8708b34-f25c-4494-a410-0d9c7d5ba1ee" containerID="9cb3473cad4ca081d7fcd7b842f6ff260e855cb5a9983963588384c92210b999" exitCode=0 Jul 29 15:30:09.490760 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.490754 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j" event={"ID":"b8708b34-f25c-4494-a410-0d9c7d5ba1ee","Type":"ContainerDied","Data":"9cb3473cad4ca081d7fcd7b842f6ff260e855cb5a9983963588384c92210b999"} Jul 29 15:30:09.490967 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.490774 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j" Jul 29 15:30:09.490967 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.490792 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j" event={"ID":"b8708b34-f25c-4494-a410-0d9c7d5ba1ee","Type":"ContainerDied","Data":"8f448a3b5ef956c0a8176e1da7ec7c0f40daf393826e105445557995eaca82e9"} Jul 29 15:30:09.502654 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.502471 2580 scope.go:117] "RemoveContainer" containerID="8d0bf9261a41a5f007871941aeca977dc61f423d2ea6220f58e5bbeb6c4ae22a" Jul 29 15:30:09.503023 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:30:09.502995 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d0bf9261a41a5f007871941aeca977dc61f423d2ea6220f58e5bbeb6c4ae22a\": container with ID starting with 8d0bf9261a41a5f007871941aeca977dc61f423d2ea6220f58e5bbeb6c4ae22a not found: ID does not exist" containerID="8d0bf9261a41a5f007871941aeca977dc61f423d2ea6220f58e5bbeb6c4ae22a" Jul 29 15:30:09.503123 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.503023 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d0bf9261a41a5f007871941aeca977dc61f423d2ea6220f58e5bbeb6c4ae22a"} err="failed to get container status \"8d0bf9261a41a5f007871941aeca977dc61f423d2ea6220f58e5bbeb6c4ae22a\": rpc error: code = NotFound desc = could not find container \"8d0bf9261a41a5f007871941aeca977dc61f423d2ea6220f58e5bbeb6c4ae22a\": container with ID starting with 8d0bf9261a41a5f007871941aeca977dc61f423d2ea6220f58e5bbeb6c4ae22a not found: ID does not exist" Jul 29 15:30:09.503123 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.503039 2580 scope.go:117] "RemoveContainer" containerID="9cb3473cad4ca081d7fcd7b842f6ff260e855cb5a9983963588384c92210b999" Jul 29 15:30:09.511148 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.511132 2580 scope.go:117] "RemoveContainer" containerID="9cb3473cad4ca081d7fcd7b842f6ff260e855cb5a9983963588384c92210b999" Jul 29 15:30:09.511407 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:30:09.511392 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cb3473cad4ca081d7fcd7b842f6ff260e855cb5a9983963588384c92210b999\": container with ID starting with 9cb3473cad4ca081d7fcd7b842f6ff260e855cb5a9983963588384c92210b999 not found: ID does not exist" containerID="9cb3473cad4ca081d7fcd7b842f6ff260e855cb5a9983963588384c92210b999" Jul 29 15:30:09.511466 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.511411 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cb3473cad4ca081d7fcd7b842f6ff260e855cb5a9983963588384c92210b999"} err="failed to get container status \"9cb3473cad4ca081d7fcd7b842f6ff260e855cb5a9983963588384c92210b999\": rpc error: code = NotFound desc = could not find container \"9cb3473cad4ca081d7fcd7b842f6ff260e855cb5a9983963588384c92210b999\": container with ID starting with 9cb3473cad4ca081d7fcd7b842f6ff260e855cb5a9983963588384c92210b999 not found: ID does not exist" Jul 29 15:30:09.513567 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.513547 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw"] Jul 29 15:30:09.519701 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.519681 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-3261e1-699d599b8d-wphnw"] Jul 29 15:30:09.529762 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.529744 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j"] Jul 29 15:30:09.535880 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.535862 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-3261e1-569cd44db6-tcz8j"] Jul 29 15:30:09.818442 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.818415 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0acec3d3-3ce1-4086-bec0-08be2669f967" path="/var/lib/kubelet/pods/0acec3d3-3ce1-4086-bec0-08be2669f967/volumes" Jul 29 15:30:09.818764 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.818666 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b5421c3-ac30-4f0e-8b91-82667e94eb25" path="/var/lib/kubelet/pods/2b5421c3-ac30-4f0e-8b91-82667e94eb25/volumes" Jul 29 15:30:09.818933 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.818920 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9abdc318-4bc8-439d-bb56-a0acb015d605" path="/var/lib/kubelet/pods/9abdc318-4bc8-439d-bb56-a0acb015d605/volumes" Jul 29 15:30:09.819230 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:09.819219 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8708b34-f25c-4494-a410-0d9c7d5ba1ee" path="/var/lib/kubelet/pods/b8708b34-f25c-4494-a410-0d9c7d5ba1ee/volumes" Jul 29 15:30:14.418437 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:14.418405 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:30:24.038737 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:24.038698 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 15:30:24.048623 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:24.048604 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-67v2n_64bb03c8-45a0-470b-a207-e1e660b6faf2/ovn-acl-logging/0.log" Jul 29 15:30:36.514912 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:36.514876 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:30:36.600074 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:36.600036 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:30:51.666882 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:51.666807 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:30:53.204538 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:53.204505 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:30:59.178894 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.178860 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:30:59.273963 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.273931 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:30:59.504133 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.504104 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:30:59.728142 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.728107 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:30:59.862419 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862349 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a4092e-swl62"] Jul 29 15:30:59.862760 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862747 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52031ac1-81e4-43d6-aa79-762f1c461b8f" containerName="cleanup" Jul 29 15:30:59.862816 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862762 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="52031ac1-81e4-43d6-aa79-762f1c461b8f" containerName="cleanup" Jul 29 15:30:59.862816 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862771 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b5421c3-ac30-4f0e-8b91-82667e94eb25" containerName="cleanup" Jul 29 15:30:59.862816 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862776 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b5421c3-ac30-4f0e-8b91-82667e94eb25" containerName="cleanup" Jul 29 15:30:59.862816 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862787 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93fea96f-35f0-4de2-8d5d-641566ff956f" containerName="cleanup" Jul 29 15:30:59.862816 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862792 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="93fea96f-35f0-4de2-8d5d-641566ff956f" containerName="cleanup" Jul 29 15:30:59.862816 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862798 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f51fa978-0c3d-4ff8-a827-91b044de33bf" containerName="cleanup" Jul 29 15:30:59.862816 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862804 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f51fa978-0c3d-4ff8-a827-91b044de33bf" containerName="cleanup" Jul 29 15:30:59.862816 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862811 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73801393-74d6-40bb-998b-faa7589273a0" containerName="cleanup" Jul 29 15:30:59.862816 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862816 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="73801393-74d6-40bb-998b-faa7589273a0" containerName="cleanup" Jul 29 15:30:59.863121 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862831 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0acec3d3-3ce1-4086-bec0-08be2669f967" containerName="payload-pre-processing" Jul 29 15:30:59.863121 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862838 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0acec3d3-3ce1-4086-bec0-08be2669f967" containerName="payload-pre-processing" Jul 29 15:30:59.863121 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862846 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8708b34-f25c-4494-a410-0d9c7d5ba1ee" containerName="payload-processing" Jul 29 15:30:59.863121 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862852 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8708b34-f25c-4494-a410-0d9c7d5ba1ee" containerName="payload-processing" Jul 29 15:30:59.863121 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862912 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8708b34-f25c-4494-a410-0d9c7d5ba1ee" containerName="payload-processing" Jul 29 15:30:59.863121 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862920 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b5421c3-ac30-4f0e-8b91-82667e94eb25" containerName="cleanup" Jul 29 15:30:59.863121 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862926 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0acec3d3-3ce1-4086-bec0-08be2669f967" containerName="payload-pre-processing" Jul 29 15:30:59.863121 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862931 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="93fea96f-35f0-4de2-8d5d-641566ff956f" containerName="cleanup" Jul 29 15:30:59.863121 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862936 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="73801393-74d6-40bb-998b-faa7589273a0" containerName="cleanup" Jul 29 15:30:59.863121 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862941 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f51fa978-0c3d-4ff8-a827-91b044de33bf" containerName="cleanup" Jul 29 15:30:59.863121 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.862947 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="52031ac1-81e4-43d6-aa79-762f1c461b8f" containerName="cleanup" Jul 29 15:30:59.866193 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.866176 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a4092e-swl62" Jul 29 15:30:59.868433 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.868410 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 29 15:30:59.868931 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.868910 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mmt6p\"" Jul 29 15:30:59.873192 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.873172 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a4092e-swl62"] Jul 29 15:30:59.923343 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:30:59.923316 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1680dc21-7828-4b55-8a1e-557133405861-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a4092e-swl62\" (UID: \"1680dc21-7828-4b55-8a1e-557133405861\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a4092e-swl62" Jul 29 15:31:00.024255 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:00.024228 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1680dc21-7828-4b55-8a1e-557133405861-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a4092e-swl62\" (UID: \"1680dc21-7828-4b55-8a1e-557133405861\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a4092e-swl62" Jul 29 15:31:00.024852 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:00.024829 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1680dc21-7828-4b55-8a1e-557133405861-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-a4092e-swl62\" (UID: \"1680dc21-7828-4b55-8a1e-557133405861\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a4092e-swl62" Jul 29 15:31:00.177777 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:00.177678 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a4092e-swl62" Jul 29 15:31:00.530538 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:00.530513 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a4092e-swl62"] Jul 29 15:31:00.531715 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:31:00.531687 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1680dc21_7828_4b55_8a1e_557133405861.slice/crio-b38b6f91d25744876c1e7c184f80c97950869511f2bd21707258e19b4346ee2e WatchSource:0}: Error finding container b38b6f91d25744876c1e7c184f80c97950869511f2bd21707258e19b4346ee2e: Status 404 returned error can't find the container with id b38b6f91d25744876c1e7c184f80c97950869511f2bd21707258e19b4346ee2e Jul 29 15:31:00.796927 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:00.796863 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a4092e-swl62" event={"ID":"1680dc21-7828-4b55-8a1e-557133405861","Type":"ContainerStarted","Data":"b38b6f91d25744876c1e7c184f80c97950869511f2bd21707258e19b4346ee2e"} Jul 29 15:31:01.803989 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:01.803950 2580 generic.go:356] "Generic (PLEG): container finished" podID="1680dc21-7828-4b55-8a1e-557133405861" containerID="52710ce0bbf1e699ef446f42f56556982f1c68730505fd63ed3cb6b72ab3f774" exitCode=0 Jul 29 15:31:01.804381 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:01.804042 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a4092e-swl62" event={"ID":"1680dc21-7828-4b55-8a1e-557133405861","Type":"ContainerDied","Data":"52710ce0bbf1e699ef446f42f56556982f1c68730505fd63ed3cb6b72ab3f774"} Jul 29 15:31:02.952045 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:02.952016 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a4092e-swl62" Jul 29 15:31:03.050247 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:03.050218 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1680dc21-7828-4b55-8a1e-557133405861-maas-api-service-ca\") pod \"1680dc21-7828-4b55-8a1e-557133405861\" (UID: \"1680dc21-7828-4b55-8a1e-557133405861\") " Jul 29 15:31:03.050521 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:03.050501 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1680dc21-7828-4b55-8a1e-557133405861-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1680dc21-7828-4b55-8a1e-557133405861" (UID: "1680dc21-7828-4b55-8a1e-557133405861"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:31:03.151707 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:03.151648 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1680dc21-7828-4b55-8a1e-557133405861-maas-api-service-ca\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:31:03.817966 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:03.817941 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a4092e-swl62" Jul 29 15:31:03.818410 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:03.818390 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a4092e-swl62" event={"ID":"1680dc21-7828-4b55-8a1e-557133405861","Type":"ContainerDied","Data":"b38b6f91d25744876c1e7c184f80c97950869511f2bd21707258e19b4346ee2e"} Jul 29 15:31:03.818472 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:03.818429 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b38b6f91d25744876c1e7c184f80c97950869511f2bd21707258e19b4346ee2e" Jul 29 15:31:10.486015 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.485984 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a4092e-29755650-9h9cj"] Jul 29 15:31:10.488546 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.488521 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-a4092e-29755650-9h9cj"] Jul 29 15:31:10.493340 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.493319 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:10.546102 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.546077 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj"] Jul 29 15:31:10.546334 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.546309 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj" podUID="6dbe1be3-6e15-44da-9d20-6d5319dde07c" containerName="payload-processing" containerID="cri-o://8b70daff12d1329fb2f342139e3d139798a098437bc29f8b863fd66c1e54d7fb" gracePeriod=30 Jul 29 15:31:10.556898 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.556876 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt"] Jul 29 15:31:10.557264 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.557222 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt" podUID="16cf7c54-77f6-45ea-91e4-327699f19c90" containerName="payload-pre-processing" containerID="cri-o://0654bb2eca210dd9bdbee9e0d81cc99577fee61783996756ba0fb4f1dc76e8d1" gracePeriod=30 Jul 29 15:31:10.583630 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.583606 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:10.684809 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.684770 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:10.733116 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.733088 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a4092e-swl62"] Jul 29 15:31:10.745400 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.745323 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-a4092e-swl62"] Jul 29 15:31:10.829080 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.829052 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj" Jul 29 15:31:10.851600 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.851580 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt" Jul 29 15:31:10.871420 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.871394 2580 generic.go:356] "Generic (PLEG): container finished" podID="6dbe1be3-6e15-44da-9d20-6d5319dde07c" containerID="8b70daff12d1329fb2f342139e3d139798a098437bc29f8b863fd66c1e54d7fb" exitCode=0 Jul 29 15:31:10.871557 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.871436 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj" event={"ID":"6dbe1be3-6e15-44da-9d20-6d5319dde07c","Type":"ContainerDied","Data":"8b70daff12d1329fb2f342139e3d139798a098437bc29f8b863fd66c1e54d7fb"} Jul 29 15:31:10.871557 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.871445 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj" Jul 29 15:31:10.871557 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.871511 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj" event={"ID":"6dbe1be3-6e15-44da-9d20-6d5319dde07c","Type":"ContainerDied","Data":"d15cedd62dea35082019be49e4f251be44070abb52fb5998ca3c18fc2b7e689e"} Jul 29 15:31:10.871557 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.871541 2580 scope.go:117] "RemoveContainer" containerID="8b70daff12d1329fb2f342139e3d139798a098437bc29f8b863fd66c1e54d7fb" Jul 29 15:31:10.871779 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.871658 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:10.873027 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.873008 2580 generic.go:356] "Generic (PLEG): container finished" podID="16cf7c54-77f6-45ea-91e4-327699f19c90" containerID="0654bb2eca210dd9bdbee9e0d81cc99577fee61783996756ba0fb4f1dc76e8d1" exitCode=0 Jul 29 15:31:10.873127 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.873077 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt" Jul 29 15:31:10.873127 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.873109 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt" event={"ID":"16cf7c54-77f6-45ea-91e4-327699f19c90","Type":"ContainerDied","Data":"0654bb2eca210dd9bdbee9e0d81cc99577fee61783996756ba0fb4f1dc76e8d1"} Jul 29 15:31:10.873210 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.873159 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt" event={"ID":"16cf7c54-77f6-45ea-91e4-327699f19c90","Type":"ContainerDied","Data":"f7c1d184f1117683f3673062f7980787c48253face49db8be64add0633f49056"} Jul 29 15:31:10.881086 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.881070 2580 scope.go:117] "RemoveContainer" containerID="8b70daff12d1329fb2f342139e3d139798a098437bc29f8b863fd66c1e54d7fb" Jul 29 15:31:10.881337 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:31:10.881313 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b70daff12d1329fb2f342139e3d139798a098437bc29f8b863fd66c1e54d7fb\": container with ID starting with 8b70daff12d1329fb2f342139e3d139798a098437bc29f8b863fd66c1e54d7fb not found: ID does not exist" containerID="8b70daff12d1329fb2f342139e3d139798a098437bc29f8b863fd66c1e54d7fb" Jul 29 15:31:10.881424 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.881344 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b70daff12d1329fb2f342139e3d139798a098437bc29f8b863fd66c1e54d7fb"} err="failed to get container status \"8b70daff12d1329fb2f342139e3d139798a098437bc29f8b863fd66c1e54d7fb\": rpc error: code = NotFound desc = could not find container \"8b70daff12d1329fb2f342139e3d139798a098437bc29f8b863fd66c1e54d7fb\": container with ID starting with 8b70daff12d1329fb2f342139e3d139798a098437bc29f8b863fd66c1e54d7fb not found: ID does not exist" Jul 29 15:31:10.881424 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.881360 2580 scope.go:117] "RemoveContainer" containerID="0654bb2eca210dd9bdbee9e0d81cc99577fee61783996756ba0fb4f1dc76e8d1" Jul 29 15:31:10.889937 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.889908 2580 scope.go:117] "RemoveContainer" containerID="0654bb2eca210dd9bdbee9e0d81cc99577fee61783996756ba0fb4f1dc76e8d1" Jul 29 15:31:10.890183 ip-10-0-132-42 kubenswrapper[2580]: E0729 15:31:10.890164 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0654bb2eca210dd9bdbee9e0d81cc99577fee61783996756ba0fb4f1dc76e8d1\": container with ID starting with 0654bb2eca210dd9bdbee9e0d81cc99577fee61783996756ba0fb4f1dc76e8d1 not found: ID does not exist" containerID="0654bb2eca210dd9bdbee9e0d81cc99577fee61783996756ba0fb4f1dc76e8d1" Jul 29 15:31:10.890233 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.890189 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0654bb2eca210dd9bdbee9e0d81cc99577fee61783996756ba0fb4f1dc76e8d1"} err="failed to get container status \"0654bb2eca210dd9bdbee9e0d81cc99577fee61783996756ba0fb4f1dc76e8d1\": rpc error: code = NotFound desc = could not find container \"0654bb2eca210dd9bdbee9e0d81cc99577fee61783996756ba0fb4f1dc76e8d1\": container with ID starting with 0654bb2eca210dd9bdbee9e0d81cc99577fee61783996756ba0fb4f1dc76e8d1 not found: ID does not exist" Jul 29 15:31:10.916114 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.916086 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6dbe1be3-6e15-44da-9d20-6d5319dde07c-plugins-config-volume\") pod \"6dbe1be3-6e15-44da-9d20-6d5319dde07c\" (UID: \"6dbe1be3-6e15-44da-9d20-6d5319dde07c\") " Jul 29 15:31:10.916219 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.916124 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tf4qp\" (UniqueName: \"kubernetes.io/projected/6dbe1be3-6e15-44da-9d20-6d5319dde07c-kube-api-access-tf4qp\") pod \"6dbe1be3-6e15-44da-9d20-6d5319dde07c\" (UID: \"6dbe1be3-6e15-44da-9d20-6d5319dde07c\") " Jul 29 15:31:10.916446 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.916420 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dbe1be3-6e15-44da-9d20-6d5319dde07c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6dbe1be3-6e15-44da-9d20-6d5319dde07c" (UID: "6dbe1be3-6e15-44da-9d20-6d5319dde07c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:31:10.916839 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.916670 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6dbe1be3-6e15-44da-9d20-6d5319dde07c-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:31:10.918599 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:10.918578 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dbe1be3-6e15-44da-9d20-6d5319dde07c-kube-api-access-tf4qp" (OuterVolumeSpecName: "kube-api-access-tf4qp") pod "6dbe1be3-6e15-44da-9d20-6d5319dde07c" (UID: "6dbe1be3-6e15-44da-9d20-6d5319dde07c"). InnerVolumeSpecName "kube-api-access-tf4qp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:31:11.017750 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:11.017677 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16cf7c54-77f6-45ea-91e4-327699f19c90-plugins-config-volume\") pod \"16cf7c54-77f6-45ea-91e4-327699f19c90\" (UID: \"16cf7c54-77f6-45ea-91e4-327699f19c90\") " Jul 29 15:31:11.017849 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:11.017756 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9qdl\" (UniqueName: \"kubernetes.io/projected/16cf7c54-77f6-45ea-91e4-327699f19c90-kube-api-access-b9qdl\") pod \"16cf7c54-77f6-45ea-91e4-327699f19c90\" (UID: \"16cf7c54-77f6-45ea-91e4-327699f19c90\") " Jul 29 15:31:11.017948 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:11.017936 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tf4qp\" (UniqueName: \"kubernetes.io/projected/6dbe1be3-6e15-44da-9d20-6d5319dde07c-kube-api-access-tf4qp\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:31:11.017996 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:11.017969 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16cf7c54-77f6-45ea-91e4-327699f19c90-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "16cf7c54-77f6-45ea-91e4-327699f19c90" (UID: "16cf7c54-77f6-45ea-91e4-327699f19c90"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:31:11.019500 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:11.019479 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16cf7c54-77f6-45ea-91e4-327699f19c90-kube-api-access-b9qdl" (OuterVolumeSpecName: "kube-api-access-b9qdl") pod "16cf7c54-77f6-45ea-91e4-327699f19c90" (UID: "16cf7c54-77f6-45ea-91e4-327699f19c90"). InnerVolumeSpecName "kube-api-access-b9qdl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:31:11.119070 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:11.119048 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/16cf7c54-77f6-45ea-91e4-327699f19c90-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:31:11.119070 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:11.119068 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9qdl\" (UniqueName: \"kubernetes.io/projected/16cf7c54-77f6-45ea-91e4-327699f19c90-kube-api-access-b9qdl\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:31:11.200656 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:11.200634 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj"] Jul 29 15:31:11.206297 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:11.206278 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a4092e-5fb8f7cb7d-k57rj"] Jul 29 15:31:11.211275 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:11.211254 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:11.219630 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:11.219611 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt"] Jul 29 15:31:11.223622 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:11.223602 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-a4092e-655d5c758d-f4hpt"] Jul 29 15:31:11.817867 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:11.817836 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1680dc21-7828-4b55-8a1e-557133405861" path="/var/lib/kubelet/pods/1680dc21-7828-4b55-8a1e-557133405861/volumes" Jul 29 15:31:11.818258 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:11.818218 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16cf7c54-77f6-45ea-91e4-327699f19c90" path="/var/lib/kubelet/pods/16cf7c54-77f6-45ea-91e4-327699f19c90/volumes" Jul 29 15:31:11.818562 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:11.818546 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dbe1be3-6e15-44da-9d20-6d5319dde07c" path="/var/lib/kubelet/pods/6dbe1be3-6e15-44da-9d20-6d5319dde07c/volumes" Jul 29 15:31:11.818908 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:11.818892 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73801393-74d6-40bb-998b-faa7589273a0" path="/var/lib/kubelet/pods/73801393-74d6-40bb-998b-faa7589273a0/volumes" Jul 29 15:31:14.443066 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:14.443032 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:14.543974 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:14.543942 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:19.050460 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:19.050422 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:19.194766 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:19.194712 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:19.469713 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:19.469631 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:19.723834 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:19.723704 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:30.720141 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:30.720106 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:30.737373 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:30.737337 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c73214-29755650-csc2r"] Jul 29 15:31:30.742643 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:30.742615 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c73214-29755650-csc2r"] Jul 29 15:31:30.786935 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:30.786902 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf"] Jul 29 15:31:30.787208 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:30.787147 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf" podUID="5d2fd091-10c2-4116-84ab-5851880c637f" containerName="payload-processing" containerID="cri-o://4f93d823a3eef00ef4729b6e3739b586c4780d36645cc889c5c7e58890d71076" gracePeriod=30 Jul 29 15:31:30.800665 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:30.800633 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q"] Jul 29 15:31:30.800915 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:30.800892 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q" podUID="938d357a-10d7-4e12-9e12-44db76aaa763" containerName="payload-pre-processing" containerID="cri-o://74191fc531a9e6ee6bdc73ab874c93c70ebe04d9df4fac6cdcd5a99df3cd4d8b" gracePeriod=30 Jul 29 15:31:30.840827 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:30.840793 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:30.908713 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:30.908667 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:30.981601 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:30.981434 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:30.999592 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:30.999560 2580 generic.go:356] "Generic (PLEG): container finished" podID="5d2fd091-10c2-4116-84ab-5851880c637f" containerID="4f93d823a3eef00ef4729b6e3739b586c4780d36645cc889c5c7e58890d71076" exitCode=0 Jul 29 15:31:30.999815 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:30.999637 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf" event={"ID":"5d2fd091-10c2-4116-84ab-5851880c637f","Type":"ContainerDied","Data":"4f93d823a3eef00ef4729b6e3739b586c4780d36645cc889c5c7e58890d71076"} Jul 29 15:31:31.001217 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:31.001194 2580 generic.go:356] "Generic (PLEG): container finished" podID="938d357a-10d7-4e12-9e12-44db76aaa763" containerID="74191fc531a9e6ee6bdc73ab874c93c70ebe04d9df4fac6cdcd5a99df3cd4d8b" exitCode=0 Jul 29 15:31:31.001319 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:31.001229 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q" event={"ID":"938d357a-10d7-4e12-9e12-44db76aaa763","Type":"ContainerDied","Data":"74191fc531a9e6ee6bdc73ab874c93c70ebe04d9df4fac6cdcd5a99df3cd4d8b"} Jul 29 15:31:31.086642 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:31.086621 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf" Jul 29 15:31:31.089410 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:31.089391 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q" Jul 29 15:31:31.187375 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:31.187342 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d2fd091-10c2-4116-84ab-5851880c637f-plugins-config-volume\") pod \"5d2fd091-10c2-4116-84ab-5851880c637f\" (UID: \"5d2fd091-10c2-4116-84ab-5851880c637f\") " Jul 29 15:31:31.187560 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:31.187407 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-282jp\" (UniqueName: \"kubernetes.io/projected/938d357a-10d7-4e12-9e12-44db76aaa763-kube-api-access-282jp\") pod \"938d357a-10d7-4e12-9e12-44db76aaa763\" (UID: \"938d357a-10d7-4e12-9e12-44db76aaa763\") " Jul 29 15:31:31.187560 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:31.187434 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/938d357a-10d7-4e12-9e12-44db76aaa763-plugins-config-volume\") pod \"938d357a-10d7-4e12-9e12-44db76aaa763\" (UID: \"938d357a-10d7-4e12-9e12-44db76aaa763\") " Jul 29 15:31:31.187560 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:31.187488 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdqqq\" (UniqueName: \"kubernetes.io/projected/5d2fd091-10c2-4116-84ab-5851880c637f-kube-api-access-qdqqq\") pod \"5d2fd091-10c2-4116-84ab-5851880c637f\" (UID: \"5d2fd091-10c2-4116-84ab-5851880c637f\") " Jul 29 15:31:31.187876 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:31.187843 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d2fd091-10c2-4116-84ab-5851880c637f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5d2fd091-10c2-4116-84ab-5851880c637f" (UID: "5d2fd091-10c2-4116-84ab-5851880c637f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:31:31.187991 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:31.187916 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/938d357a-10d7-4e12-9e12-44db76aaa763-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "938d357a-10d7-4e12-9e12-44db76aaa763" (UID: "938d357a-10d7-4e12-9e12-44db76aaa763"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:31:31.189581 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:31.189558 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d2fd091-10c2-4116-84ab-5851880c637f-kube-api-access-qdqqq" (OuterVolumeSpecName: "kube-api-access-qdqqq") pod "5d2fd091-10c2-4116-84ab-5851880c637f" (UID: "5d2fd091-10c2-4116-84ab-5851880c637f"). InnerVolumeSpecName "kube-api-access-qdqqq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:31:31.189668 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:31.189581 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/938d357a-10d7-4e12-9e12-44db76aaa763-kube-api-access-282jp" (OuterVolumeSpecName: "kube-api-access-282jp") pod "938d357a-10d7-4e12-9e12-44db76aaa763" (UID: "938d357a-10d7-4e12-9e12-44db76aaa763"). InnerVolumeSpecName "kube-api-access-282jp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:31:31.289145 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:31.289110 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdqqq\" (UniqueName: \"kubernetes.io/projected/5d2fd091-10c2-4116-84ab-5851880c637f-kube-api-access-qdqqq\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:31:31.289145 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:31.289137 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d2fd091-10c2-4116-84ab-5851880c637f-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:31:31.289145 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:31.289146 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-282jp\" (UniqueName: \"kubernetes.io/projected/938d357a-10d7-4e12-9e12-44db76aaa763-kube-api-access-282jp\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:31:31.289145 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:31.289156 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/938d357a-10d7-4e12-9e12-44db76aaa763-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:31:31.407120 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:31.407092 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:31.818872 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:31.818831 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f51fa978-0c3d-4ff8-a827-91b044de33bf" path="/var/lib/kubelet/pods/f51fa978-0c3d-4ff8-a827-91b044de33bf/volumes" Jul 29 15:31:32.009605 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:32.009573 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf" event={"ID":"5d2fd091-10c2-4116-84ab-5851880c637f","Type":"ContainerDied","Data":"b45d1360c775fa8eff447dde9e3bb509ad0f23c2511a3114087f780345f7c5ab"} Jul 29 15:31:32.009806 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:32.009639 2580 scope.go:117] "RemoveContainer" containerID="4f93d823a3eef00ef4729b6e3739b586c4780d36645cc889c5c7e58890d71076" Jul 29 15:31:32.009806 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:32.009586 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf" Jul 29 15:31:32.011060 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:32.011042 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q" Jul 29 15:31:32.011182 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:32.011086 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q" event={"ID":"938d357a-10d7-4e12-9e12-44db76aaa763","Type":"ContainerDied","Data":"888d785c2a3885ebad5c7476f5ba859315a104633ba2a670b1ba30785231a4dc"} Jul 29 15:31:32.018733 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:32.018701 2580 scope.go:117] "RemoveContainer" containerID="74191fc531a9e6ee6bdc73ab874c93c70ebe04d9df4fac6cdcd5a99df3cd4d8b" Jul 29 15:31:32.031816 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:32.031793 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf"] Jul 29 15:31:32.035369 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:32.035352 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-c73214-687b78dbc4-jf5wf"] Jul 29 15:31:32.052740 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:32.052697 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q"] Jul 29 15:31:32.057279 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:32.057253 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-c73214-7d68c66dcb-fzf2q"] Jul 29 15:31:33.819324 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:33.819286 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d2fd091-10c2-4116-84ab-5851880c637f" path="/var/lib/kubelet/pods/5d2fd091-10c2-4116-84ab-5851880c637f/volumes" Jul 29 15:31:33.819922 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:33.819901 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="938d357a-10d7-4e12-9e12-44db76aaa763" path="/var/lib/kubelet/pods/938d357a-10d7-4e12-9e12-44db76aaa763/volumes" Jul 29 15:31:37.603286 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:37.603253 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:37.701994 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:37.701963 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:37.961252 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:37.961176 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:38.214389 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:38.214318 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:48.909560 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:48.909491 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:48.910466 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:48.910422 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-93e799-29755650-hdp7b"] Jul 29 15:31:48.915077 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:48.915028 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-93e799-29755650-hdp7b"] Jul 29 15:31:48.939407 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:48.939374 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8"] Jul 29 15:31:48.939690 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:48.939641 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8" podUID="14f15644-65fc-43d8-9e52-b6ee74ef108d" containerName="payload-processing" containerID="cri-o://bc55dd0b45c37e291b8d0940db0cc16d2b315a0a3c1deb720ffe4eb211ba1295" gracePeriod=30 Jul 29 15:31:48.951111 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:48.951086 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d"] Jul 29 15:31:48.951708 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:48.951318 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d" podUID="80d9f2c0-5975-4b05-8413-8a324b252448" containerName="payload-pre-processing" containerID="cri-o://ccf156d7a518d8007523060d80dfb470afa7602affd9dc3320f653845150ff37" gracePeriod=30 Jul 29 15:31:48.986742 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:48.983205 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:49.061917 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:49.061887 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:49.122065 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:49.122033 2580 generic.go:356] "Generic (PLEG): container finished" podID="14f15644-65fc-43d8-9e52-b6ee74ef108d" containerID="bc55dd0b45c37e291b8d0940db0cc16d2b315a0a3c1deb720ffe4eb211ba1295" exitCode=0 Jul 29 15:31:49.122482 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:49.122214 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8" event={"ID":"14f15644-65fc-43d8-9e52-b6ee74ef108d","Type":"ContainerDied","Data":"bc55dd0b45c37e291b8d0940db0cc16d2b315a0a3c1deb720ffe4eb211ba1295"} Jul 29 15:31:49.125342 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:49.125215 2580 generic.go:356] "Generic (PLEG): container finished" podID="80d9f2c0-5975-4b05-8413-8a324b252448" containerID="ccf156d7a518d8007523060d80dfb470afa7602affd9dc3320f653845150ff37" exitCode=0 Jul 29 15:31:49.125342 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:49.125284 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d" event={"ID":"80d9f2c0-5975-4b05-8413-8a324b252448","Type":"ContainerDied","Data":"ccf156d7a518d8007523060d80dfb470afa7602affd9dc3320f653845150ff37"} Jul 29 15:31:49.228819 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:49.228796 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d" Jul 29 15:31:49.231836 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:49.231814 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8" Jul 29 15:31:49.334599 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:49.334557 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80d9f2c0-5975-4b05-8413-8a324b252448-plugins-config-volume\") pod \"80d9f2c0-5975-4b05-8413-8a324b252448\" (UID: \"80d9f2c0-5975-4b05-8413-8a324b252448\") " Jul 29 15:31:49.334811 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:49.334619 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wphk2\" (UniqueName: \"kubernetes.io/projected/14f15644-65fc-43d8-9e52-b6ee74ef108d-kube-api-access-wphk2\") pod \"14f15644-65fc-43d8-9e52-b6ee74ef108d\" (UID: \"14f15644-65fc-43d8-9e52-b6ee74ef108d\") " Jul 29 15:31:49.334811 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:49.334649 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14f15644-65fc-43d8-9e52-b6ee74ef108d-plugins-config-volume\") pod \"14f15644-65fc-43d8-9e52-b6ee74ef108d\" (UID: \"14f15644-65fc-43d8-9e52-b6ee74ef108d\") " Jul 29 15:31:49.334811 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:49.334685 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9sn2k\" (UniqueName: \"kubernetes.io/projected/80d9f2c0-5975-4b05-8413-8a324b252448-kube-api-access-9sn2k\") pod \"80d9f2c0-5975-4b05-8413-8a324b252448\" (UID: \"80d9f2c0-5975-4b05-8413-8a324b252448\") " Jul 29 15:31:49.335032 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:49.335008 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80d9f2c0-5975-4b05-8413-8a324b252448-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "80d9f2c0-5975-4b05-8413-8a324b252448" (UID: "80d9f2c0-5975-4b05-8413-8a324b252448"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:31:49.335087 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:49.335032 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14f15644-65fc-43d8-9e52-b6ee74ef108d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "14f15644-65fc-43d8-9e52-b6ee74ef108d" (UID: "14f15644-65fc-43d8-9e52-b6ee74ef108d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 15:31:49.336784 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:49.336759 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14f15644-65fc-43d8-9e52-b6ee74ef108d-kube-api-access-wphk2" (OuterVolumeSpecName: "kube-api-access-wphk2") pod "14f15644-65fc-43d8-9e52-b6ee74ef108d" (UID: "14f15644-65fc-43d8-9e52-b6ee74ef108d"). InnerVolumeSpecName "kube-api-access-wphk2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:31:49.336784 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:49.336767 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80d9f2c0-5975-4b05-8413-8a324b252448-kube-api-access-9sn2k" (OuterVolumeSpecName: "kube-api-access-9sn2k") pod "80d9f2c0-5975-4b05-8413-8a324b252448" (UID: "80d9f2c0-5975-4b05-8413-8a324b252448"). InnerVolumeSpecName "kube-api-access-9sn2k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 15:31:49.435399 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:49.435366 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sn2k\" (UniqueName: \"kubernetes.io/projected/80d9f2c0-5975-4b05-8413-8a324b252448-kube-api-access-9sn2k\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:31:49.435399 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:49.435397 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80d9f2c0-5975-4b05-8413-8a324b252448-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:31:49.435399 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:49.435406 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wphk2\" (UniqueName: \"kubernetes.io/projected/14f15644-65fc-43d8-9e52-b6ee74ef108d-kube-api-access-wphk2\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:31:49.435627 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:49.435415 2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14f15644-65fc-43d8-9e52-b6ee74ef108d-plugins-config-volume\") on node \"ip-10-0-132-42.ec2.internal\" DevicePath \"\"" Jul 29 15:31:49.818990 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:49.818958 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52031ac1-81e4-43d6-aa79-762f1c461b8f" path="/var/lib/kubelet/pods/52031ac1-81e4-43d6-aa79-762f1c461b8f/volumes" Jul 29 15:31:49.844538 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:49.844509 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:50.132681 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:50.132606 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d" Jul 29 15:31:50.132681 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:50.132636 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d" event={"ID":"80d9f2c0-5975-4b05-8413-8a324b252448","Type":"ContainerDied","Data":"08c217217e3a4c92fd479cf3bbe13739a57c80e1a09110a3400d297275b38415"} Jul 29 15:31:50.133165 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:50.132708 2580 scope.go:117] "RemoveContainer" containerID="ccf156d7a518d8007523060d80dfb470afa7602affd9dc3320f653845150ff37" Jul 29 15:31:50.133969 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:50.133950 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8" Jul 29 15:31:50.134026 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:50.133991 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8" event={"ID":"14f15644-65fc-43d8-9e52-b6ee74ef108d","Type":"ContainerDied","Data":"1efe4e06cbdcd7d81da3a4c3b3dbbd99d7a5429cbd06a0edbe98d8d176de5200"} Jul 29 15:31:50.141588 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:50.141565 2580 scope.go:117] "RemoveContainer" containerID="bc55dd0b45c37e291b8d0940db0cc16d2b315a0a3c1deb720ffe4eb211ba1295" Jul 29 15:31:50.152436 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:50.152416 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8"] Jul 29 15:31:50.156059 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:50.156040 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-93e799-6b7d69875d-s46z8"] Jul 29 15:31:50.165680 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:50.165651 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d"] Jul 29 15:31:50.167917 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:50.167898 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-93e799-6868fb6cdd-7bt8d"] Jul 29 15:31:50.594552 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:50.594519 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:51.817659 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:51.817619 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14f15644-65fc-43d8-9e52-b6ee74ef108d" path="/var/lib/kubelet/pods/14f15644-65fc-43d8-9e52-b6ee74ef108d/volumes" Jul 29 15:31:51.818164 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:51.818150 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80d9f2c0-5975-4b05-8413-8a324b252448" path="/var/lib/kubelet/pods/80d9f2c0-5975-4b05-8413-8a324b252448/volumes" Jul 29 15:31:55.567064 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:55.567028 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:55.668912 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:55.668867 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:55.823967 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:55.823893 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:31:56.039555 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:31:56.039524 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:32:01.924348 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:01.924316 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-f8ljt_186209d4-1983-42d8-80d8-246f3a7b3fad/manager/0.log" Jul 29 15:32:02.037190 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:02.037161 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-674rf_35a7955b-5872-44c6-9237-62a0f95fe24c/manager/0.log" Jul 29 15:32:02.409431 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:02.409401 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-mxwh6_637fd95f-e250-422b-9132-580af39a1611/manager/0.log" Jul 29 15:32:02.527972 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:02.527945 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65f9f6cd8c-78t5l_2f5da262-cc15-4b81-b450-11badbd13afd/manager/0.log" Jul 29 15:32:02.536065 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:02.536045 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65f9f6cd8c-78t5l_2f5da262-cc15-4b81-b450-11badbd13afd/copy-manifests/0.log" Jul 29 15:32:03.269685 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:03.269657 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29755620-t56f7_868e3483-c702-4028-ba6d-daf8254d4eb1/cleanup/0.log" Jul 29 15:32:03.384199 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:03.384171 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29755635-rkn2r_93fcc4e2-d3e1-47e2-be9e-480081fda165/cleanup/0.log" Jul 29 15:32:03.498443 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:03.498419 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29755650-654vm_93fea96f-35f0-4de2-8d5d-641566ff956f/cleanup/0.log" Jul 29 15:32:04.389558 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:04.389533 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq_4d060c49-d966-486c-bf4e-35b6c3573e0a/util/0.log" Jul 29 15:32:04.396812 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:04.396794 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq_4d060c49-d966-486c-bf4e-35b6c3573e0a/pull/0.log" Jul 29 15:32:04.403860 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:04.403842 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq_4d060c49-d966-486c-bf4e-35b6c3573e0a/extract/0.log" Jul 29 15:32:04.519043 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:04.519013 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp_7936a6a4-72ef-49fe-b75d-48e925e0e807/util/0.log" Jul 29 15:32:04.526666 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:04.526647 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp_7936a6a4-72ef-49fe-b75d-48e925e0e807/pull/0.log" Jul 29 15:32:04.534079 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:04.534064 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp_7936a6a4-72ef-49fe-b75d-48e925e0e807/extract/0.log" Jul 29 15:32:04.646834 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:04.646773 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls_751fa8fe-b9a5-4026-bd77-f38f73957032/util/0.log" Jul 29 15:32:04.654744 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:04.654712 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls_751fa8fe-b9a5-4026-bd77-f38f73957032/pull/0.log" Jul 29 15:32:04.661952 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:04.661933 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls_751fa8fe-b9a5-4026-bd77-f38f73957032/extract/0.log" Jul 29 15:32:04.779154 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:04.779136 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s_8c3f17c4-f6be-4645-b934-61a6a5499b6d/util/0.log" Jul 29 15:32:04.786189 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:04.786170 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s_8c3f17c4-f6be-4645-b934-61a6a5499b6d/pull/0.log" Jul 29 15:32:04.793266 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:04.793245 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s_8c3f17c4-f6be-4645-b934-61a6a5499b6d/extract/0.log" Jul 29 15:32:05.045283 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:05.045260 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-rjtbg_d61df5ab-7566-485f-9ea1-4e181286254a/manager/0.log" Jul 29 15:32:05.520972 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:05.520943 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-kwlph_6790323f-ca7d-40af-8552-8ca4d350078a/manager/0.log" Jul 29 15:32:06.237366 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:06.237336 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-kq4pm_1c84156a-763b-44a5-bbb7-21673d1a9a51/discovery/0.log" Jul 29 15:32:06.347811 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:06.347788 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68ffcd8f6-t4pfh_11d59862-3e7f-4c04-9ca7-fe8cacecadf5/kube-auth-proxy/0.log" Jul 29 15:32:06.696438 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:06.696396 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-g5ttp_8fb9e0c2-fa8a-418c-94c8-94f7c8222b21/payload-pre-processing/0.log" Jul 29 15:32:06.815041 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:06.815019 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-45bsl_1695ee5c-3411-48e2-85e7-f11d527f6f5e/payload-processing/0.log" Jul 29 15:32:07.387630 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:07.387601 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-2qzhd_a7eeb41e-6c1c-4621-b85f-90068071b442/main/0.log" Jul 29 15:32:07.498405 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:07.498381 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-wpnfr_171f57a0-37f5-4b66-b8c7-dea91ddd1437/main/0.log" Jul 29 15:32:13.652060 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:13.652022 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:32:14.067843 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:14.067807 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:32:16.757087 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:16.757018 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tfbhf_3d2a7a6e-aa3e-44c4-820e-c46d39b7e6f9/global-pull-secret-syncer/0.log" Jul 29 15:32:16.844552 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:16.844528 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" Jul 29 15:32:16.929149 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:16.929127 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-42.ec2.internal_ba036abba3605af2a4be113b0020ec88/haproxy/0.log" Jul 29 15:32:21.567254 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:21.567220 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq_4d060c49-d966-486c-bf4e-35b6c3573e0a/extract/0.log" Jul 29 15:32:21.587196 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:21.587171 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq_4d060c49-d966-486c-bf4e-35b6c3573e0a/util/0.log" Jul 29 15:32:21.605631 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:21.605603 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z57gq_4d060c49-d966-486c-bf4e-35b6c3573e0a/pull/0.log" Jul 29 15:32:21.628539 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:21.628499 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp_7936a6a4-72ef-49fe-b75d-48e925e0e807/extract/0.log" Jul 29 15:32:21.647014 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:21.646988 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp_7936a6a4-72ef-49fe-b75d-48e925e0e807/util/0.log" Jul 29 15:32:21.665845 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:21.665810 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0qp9wp_7936a6a4-72ef-49fe-b75d-48e925e0e807/pull/0.log" Jul 29 15:32:21.688600 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:21.688552 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls_751fa8fe-b9a5-4026-bd77-f38f73957032/extract/0.log" Jul 29 15:32:21.708946 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:21.708917 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls_751fa8fe-b9a5-4026-bd77-f38f73957032/util/0.log" Jul 29 15:32:21.726478 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:21.726450 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fq8ls_751fa8fe-b9a5-4026-bd77-f38f73957032/pull/0.log" Jul 29 15:32:21.751633 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:21.751604 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s_8c3f17c4-f6be-4645-b934-61a6a5499b6d/extract/0.log" Jul 29 15:32:21.769034 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:21.769010 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s_8c3f17c4-f6be-4645-b934-61a6a5499b6d/util/0.log" Jul 29 15:32:21.791249 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:21.791223 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1kr29s_8c3f17c4-f6be-4645-b934-61a6a5499b6d/pull/0.log" Jul 29 15:32:22.265484 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:22.265456 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-rjtbg_d61df5ab-7566-485f-9ea1-4e181286254a/manager/0.log" Jul 29 15:32:22.533533 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:22.533450 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-kwlph_6790323f-ca7d-40af-8552-8ca4d350078a/manager/0.log" Jul 29 15:32:24.175488 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:24.175455 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_af869d30-2596-4b43-a83c-6c8d0e0e332f/alertmanager/0.log" Jul 29 15:32:24.196879 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:24.196849 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_af869d30-2596-4b43-a83c-6c8d0e0e332f/config-reloader/0.log" Jul 29 15:32:24.217665 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:24.217643 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_af869d30-2596-4b43-a83c-6c8d0e0e332f/kube-rbac-proxy-web/0.log" Jul 29 15:32:24.237289 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:24.237267 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_af869d30-2596-4b43-a83c-6c8d0e0e332f/kube-rbac-proxy/0.log" Jul 29 15:32:24.259898 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:24.259868 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_af869d30-2596-4b43-a83c-6c8d0e0e332f/kube-rbac-proxy-metric/0.log" Jul 29 15:32:24.281169 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:24.281147 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_af869d30-2596-4b43-a83c-6c8d0e0e332f/prom-label-proxy/0.log" Jul 29 15:32:24.307735 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:24.307704 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_af869d30-2596-4b43-a83c-6c8d0e0e332f/init-config-reloader/0.log" Jul 29 15:32:24.476994 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:24.476918 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-ttcjn_716f191c-64ee-4690-b427-76f378fd3484/monitoring-plugin/0.log" Jul 29 15:32:24.673862 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:24.673827 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kqphh_4396d907-2db8-4122-9e06-0c84c2977b7d/node-exporter/0.log" Jul 29 15:32:24.716866 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:24.716841 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kqphh_4396d907-2db8-4122-9e06-0c84c2977b7d/kube-rbac-proxy/0.log" Jul 29 15:32:24.741251 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:24.741162 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kqphh_4396d907-2db8-4122-9e06-0c84c2977b7d/init-textfile/0.log" Jul 29 15:32:24.770798 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:24.770773 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-27chd_f4162058-cbae-40d7-b6c2-e2a3133fd372/kube-rbac-proxy-main/0.log" Jul 29 15:32:24.790859 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:24.790833 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-27chd_f4162058-cbae-40d7-b6c2-e2a3133fd372/kube-rbac-proxy-self/0.log" Jul 29 15:32:24.815562 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:24.815534 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-27chd_f4162058-cbae-40d7-b6c2-e2a3133fd372/openshift-state-metrics/0.log" Jul 29 15:32:25.075035 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:25.075006 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d8d6fc794-xfmrn_7eac229a-af90-4946-88a6-4c776c96ee51/telemeter-client/0.log" Jul 29 15:32:25.099805 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:25.099773 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d8d6fc794-xfmrn_7eac229a-af90-4946-88a6-4c776c96ee51/reload/0.log" Jul 29 15:32:25.119837 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:25.119809 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d8d6fc794-xfmrn_7eac229a-af90-4946-88a6-4c776c96ee51/kube-rbac-proxy/0.log" Jul 29 15:32:26.755507 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.755469 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n"] Jul 29 15:32:26.756109 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.756088 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14f15644-65fc-43d8-9e52-b6ee74ef108d" containerName="payload-processing" Jul 29 15:32:26.756188 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.756113 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="14f15644-65fc-43d8-9e52-b6ee74ef108d" containerName="payload-processing" Jul 29 15:32:26.756188 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.756138 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80d9f2c0-5975-4b05-8413-8a324b252448" containerName="payload-pre-processing" Jul 29 15:32:26.756188 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.756148 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="80d9f2c0-5975-4b05-8413-8a324b252448" containerName="payload-pre-processing" Jul 29 15:32:26.756188 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.756159 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1680dc21-7828-4b55-8a1e-557133405861" containerName="revoke-keys" Jul 29 15:32:26.756188 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.756168 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1680dc21-7828-4b55-8a1e-557133405861" containerName="revoke-keys" Jul 29 15:32:26.756188 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.756182 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d2fd091-10c2-4116-84ab-5851880c637f" containerName="payload-processing" Jul 29 15:32:26.756546 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.756191 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d2fd091-10c2-4116-84ab-5851880c637f" containerName="payload-processing" Jul 29 15:32:26.756546 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.756201 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dbe1be3-6e15-44da-9d20-6d5319dde07c" containerName="payload-processing" Jul 29 15:32:26.756546 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.756210 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dbe1be3-6e15-44da-9d20-6d5319dde07c" containerName="payload-processing" Jul 29 15:32:26.756546 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.756225 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16cf7c54-77f6-45ea-91e4-327699f19c90" containerName="payload-pre-processing" Jul 29 15:32:26.756546 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.756234 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="16cf7c54-77f6-45ea-91e4-327699f19c90" containerName="payload-pre-processing" Jul 29 15:32:26.756546 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.756250 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="938d357a-10d7-4e12-9e12-44db76aaa763" containerName="payload-pre-processing" Jul 29 15:32:26.756546 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.756258 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="938d357a-10d7-4e12-9e12-44db76aaa763" containerName="payload-pre-processing" Jul 29 15:32:26.756546 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.756361 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="938d357a-10d7-4e12-9e12-44db76aaa763" containerName="payload-pre-processing" Jul 29 15:32:26.756546 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.756374 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="16cf7c54-77f6-45ea-91e4-327699f19c90" containerName="payload-pre-processing" Jul 29 15:32:26.756546 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.756388 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="80d9f2c0-5975-4b05-8413-8a324b252448" containerName="payload-pre-processing" Jul 29 15:32:26.756546 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.756399 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="14f15644-65fc-43d8-9e52-b6ee74ef108d" containerName="payload-processing" Jul 29 15:32:26.756546 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.756408 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d2fd091-10c2-4116-84ab-5851880c637f" containerName="payload-processing" Jul 29 15:32:26.756546 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.756416 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dbe1be3-6e15-44da-9d20-6d5319dde07c" containerName="payload-processing" Jul 29 15:32:26.756546 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.756427 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1680dc21-7828-4b55-8a1e-557133405861" containerName="revoke-keys" Jul 29 15:32:26.761381 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.761359 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" Jul 29 15:32:26.763503 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.763486 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mh6mr\"/\"default-dockercfg-t5t7t\"" Jul 29 15:32:26.763625 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.763599 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mh6mr\"/\"openshift-service-ca.crt\"" Jul 29 15:32:26.763995 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.763978 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mh6mr\"/\"kube-root-ca.crt\"" Jul 29 15:32:26.773619 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.773599 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n"] Jul 29 15:32:26.954271 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.954240 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87ctx\" (UniqueName: \"kubernetes.io/projected/fbb26e3c-0072-4164-9070-643e6212ef19-kube-api-access-87ctx\") pod \"perf-node-gather-daemonset-kk52n\" (UID: \"fbb26e3c-0072-4164-9070-643e6212ef19\") " pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" Jul 29 15:32:26.954455 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.954302 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fbb26e3c-0072-4164-9070-643e6212ef19-podres\") pod \"perf-node-gather-daemonset-kk52n\" (UID: \"fbb26e3c-0072-4164-9070-643e6212ef19\") " pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" Jul 29 15:32:26.954455 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.954398 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fbb26e3c-0072-4164-9070-643e6212ef19-sys\") pod \"perf-node-gather-daemonset-kk52n\" (UID: \"fbb26e3c-0072-4164-9070-643e6212ef19\") " pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" Jul 29 15:32:26.954455 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.954412 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fbb26e3c-0072-4164-9070-643e6212ef19-proc\") pod \"perf-node-gather-daemonset-kk52n\" (UID: \"fbb26e3c-0072-4164-9070-643e6212ef19\") " pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" Jul 29 15:32:26.954455 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:26.954432 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fbb26e3c-0072-4164-9070-643e6212ef19-lib-modules\") pod \"perf-node-gather-daemonset-kk52n\" (UID: \"fbb26e3c-0072-4164-9070-643e6212ef19\") " pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" Jul 29 15:32:27.055145 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:27.055066 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fbb26e3c-0072-4164-9070-643e6212ef19-podres\") pod \"perf-node-gather-daemonset-kk52n\" (UID: \"fbb26e3c-0072-4164-9070-643e6212ef19\") " pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" Jul 29 15:32:27.055145 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:27.055116 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fbb26e3c-0072-4164-9070-643e6212ef19-sys\") pod \"perf-node-gather-daemonset-kk52n\" (UID: \"fbb26e3c-0072-4164-9070-643e6212ef19\") " pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" Jul 29 15:32:27.055145 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:27.055132 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fbb26e3c-0072-4164-9070-643e6212ef19-proc\") pod \"perf-node-gather-daemonset-kk52n\" (UID: \"fbb26e3c-0072-4164-9070-643e6212ef19\") " pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" Jul 29 15:32:27.055370 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:27.055156 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fbb26e3c-0072-4164-9070-643e6212ef19-lib-modules\") pod \"perf-node-gather-daemonset-kk52n\" (UID: \"fbb26e3c-0072-4164-9070-643e6212ef19\") " pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" Jul 29 15:32:27.055370 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:27.055190 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87ctx\" (UniqueName: \"kubernetes.io/projected/fbb26e3c-0072-4164-9070-643e6212ef19-kube-api-access-87ctx\") pod \"perf-node-gather-daemonset-kk52n\" (UID: \"fbb26e3c-0072-4164-9070-643e6212ef19\") " pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" Jul 29 15:32:27.055370 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:27.055200 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fbb26e3c-0072-4164-9070-643e6212ef19-sys\") pod \"perf-node-gather-daemonset-kk52n\" (UID: \"fbb26e3c-0072-4164-9070-643e6212ef19\") " pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" Jul 29 15:32:27.055370 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:27.055218 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fbb26e3c-0072-4164-9070-643e6212ef19-proc\") pod \"perf-node-gather-daemonset-kk52n\" (UID: \"fbb26e3c-0072-4164-9070-643e6212ef19\") " pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" Jul 29 15:32:27.055370 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:27.055226 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fbb26e3c-0072-4164-9070-643e6212ef19-podres\") pod \"perf-node-gather-daemonset-kk52n\" (UID: \"fbb26e3c-0072-4164-9070-643e6212ef19\") " pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" Jul 29 15:32:27.055370 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:27.055309 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fbb26e3c-0072-4164-9070-643e6212ef19-lib-modules\") pod \"perf-node-gather-daemonset-kk52n\" (UID: \"fbb26e3c-0072-4164-9070-643e6212ef19\") " pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" Jul 29 15:32:27.062881 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:27.062861 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87ctx\" (UniqueName: \"kubernetes.io/projected/fbb26e3c-0072-4164-9070-643e6212ef19-kube-api-access-87ctx\") pod \"perf-node-gather-daemonset-kk52n\" (UID: \"fbb26e3c-0072-4164-9070-643e6212ef19\") " pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" Jul 29 15:32:27.071400 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:27.071382 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" Jul 29 15:32:27.198933 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:27.198906 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n"] Jul 29 15:32:27.200771 ip-10-0-132-42 kubenswrapper[2580]: W0729 15:32:27.200741 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfbb26e3c_0072_4164_9070_643e6212ef19.slice/crio-4fb461e8aedd70c94d613b2a7d93e4a2404827e4014f3a0fba7be39bf0f3a868 WatchSource:0}: Error finding container 4fb461e8aedd70c94d613b2a7d93e4a2404827e4014f3a0fba7be39bf0f3a868: Status 404 returned error can't find the container with id 4fb461e8aedd70c94d613b2a7d93e4a2404827e4014f3a0fba7be39bf0f3a868 Jul 29 15:32:27.349316 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:27.349233 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" event={"ID":"fbb26e3c-0072-4164-9070-643e6212ef19","Type":"ContainerStarted","Data":"0d30b5f6a3beaf286a0a7e5f599b298c332fc3a3b8e5be46b37fc582b8a8759e"} Jul 29 15:32:27.349316 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:27.349292 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" event={"ID":"fbb26e3c-0072-4164-9070-643e6212ef19","Type":"ContainerStarted","Data":"4fb461e8aedd70c94d613b2a7d93e4a2404827e4014f3a0fba7be39bf0f3a868"} Jul 29 15:32:27.349499 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:27.349331 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" Jul 29 15:32:27.368698 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:27.368649 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" podStartSLOduration=1.368634799 podStartE2EDuration="1.368634799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:32:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:32:27.367664173 +0000 UTC m=+3424.095813363" watchObservedRunningTime="2026-07-29 15:32:27.368634799 +0000 UTC m=+3424.096783990" Jul 29 15:32:27.529789 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:27.529751 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-84f665d64b-vrrcr_aa91c527-597c-4d17-b213-d87c5273cfc0/console/0.log" Jul 29 15:32:27.560164 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:27.560126 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-vwwj2_16bddc73-bd54-4f98-9041-5b0848562b4a/download-server/0.log" Jul 29 15:32:29.118459 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:29.118427 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hm2lz_57cd5914-9960-4c9f-a4f0-02069a80d502/dns/0.log" Jul 29 15:32:29.185106 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:29.185082 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hm2lz_57cd5914-9960-4c9f-a4f0-02069a80d502/kube-rbac-proxy/0.log" Jul 29 15:32:29.228383 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:29.228359 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-krcjx_a1e60e32-5fbd-46a4-9458-fab72b4b4b68/dns-node-resolver/0.log" Jul 29 15:32:29.732604 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:29.732576 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-847ddf4c6-zlbq6_814fcac9-b1b5-4d9f-9146-d06195946b47/registry/0.log" Jul 29 15:32:29.799527 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:29.799504 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gxzdh_0af98f75-498c-4059-924a-daa3ab0ebfbc/node-ca/0.log" Jul 29 15:32:30.821031 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:30.820999 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-kq4pm_1c84156a-763b-44a5-bbb7-21673d1a9a51/discovery/0.log" Jul 29 15:32:30.840321 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:30.840293 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68ffcd8f6-t4pfh_11d59862-3e7f-4c04-9ca7-fe8cacecadf5/kube-auth-proxy/0.log" Jul 29 15:32:30.950443 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:30.950416 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-g5ttp_8fb9e0c2-fa8a-418c-94c8-94f7c8222b21/payload-pre-processing/0.log" Jul 29 15:32:30.977576 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:30.977553 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-45bsl_1695ee5c-3411-48e2-85e7-f11d527f6f5e/payload-processing/0.log" Jul 29 15:32:31.756106 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:31.756082 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4wglj_7e841cbf-1150-4f3c-9c60-97f0c2efa9e2/serve-healthcheck-canary/0.log" Jul 29 15:32:32.370492 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:32.370460 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t9mkp_407b965c-f652-4b4b-b8be-0940e599da7b/kube-rbac-proxy/0.log" Jul 29 15:32:32.387335 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:32.387312 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t9mkp_407b965c-f652-4b4b-b8be-0940e599da7b/exporter/0.log" Jul 29 15:32:32.405128 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:32.405110 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t9mkp_407b965c-f652-4b4b-b8be-0940e599da7b/extractor/0.log" Jul 29 15:32:33.364830 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:33.364804 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mh6mr/perf-node-gather-daemonset-kk52n" Jul 29 15:32:34.359457 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:34.359429 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-f8ljt_186209d4-1983-42d8-80d8-246f3a7b3fad/manager/0.log" Jul 29 15:32:34.555652 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:34.555623 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-674rf_35a7955b-5872-44c6-9237-62a0f95fe24c/manager/0.log" Jul 29 15:32:34.789713 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:34.789661 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-mxwh6_637fd95f-e250-422b-9132-580af39a1611/manager/0.log" Jul 29 15:32:34.811296 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:34.811272 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65f9f6cd8c-78t5l_2f5da262-cc15-4b81-b450-11badbd13afd/manager/0.log" Jul 29 15:32:34.812367 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:34.812348 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:32:34.827309 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:34.827288 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65f9f6cd8c-78t5l_2f5da262-cc15-4b81-b450-11badbd13afd/copy-manifests/0.log" Jul 29 15:32:36.271084 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:36.271056 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-kq4pm_1c84156a-763b-44a5-bbb7-21673d1a9a51/discovery/0.log" Jul 29 15:32:36.287913 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:36.287887 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68ffcd8f6-t4pfh_11d59862-3e7f-4c04-9ca7-fe8cacecadf5/kube-auth-proxy/0.log" Jul 29 15:32:36.379268 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:36.379239 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-g5ttp_8fb9e0c2-fa8a-418c-94c8-94f7c8222b21/payload-pre-processing/0.log" Jul 29 15:32:36.397316 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:36.397291 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-45bsl_1695ee5c-3411-48e2-85e7-f11d527f6f5e/payload-processing/0.log" Jul 29 15:32:36.442033 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:36.442010 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-prjd2_b6cc3a03-14f5-44d1-9595-8f5942242b37/openshift-lws-operator/0.log" Jul 29 15:32:36.988592 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:36.988518 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-99w5n_36c6b7d1-eca5-4c89-a812-ea83e6ba33a7/konnectivity-agent/0.log" limit=1024 Jul 29 15:32:37.864414 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:37.864383 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-kq4pm_1c84156a-763b-44a5-bbb7-21673d1a9a51/discovery/0.log" Jul 29 15:32:37.880341 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:37.880298 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68ffcd8f6-t4pfh_11d59862-3e7f-4c04-9ca7-fe8cacecadf5/kube-auth-proxy/0.log" Jul 29 15:32:37.974173 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:37.974146 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-g5ttp_8fb9e0c2-fa8a-418c-94c8-94f7c8222b21/payload-pre-processing/0.log" Jul 29 15:32:37.994835 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:37.994810 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-45bsl_1695ee5c-3411-48e2-85e7-f11d527f6f5e/payload-processing/0.log" Jul 29 15:32:42.533019 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:42.532988 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n84x4_7c5930f2-ccc2-476a-8d31-9e85d2735b7a/kube-multus-additional-cni-plugins/0.log" Jul 29 15:32:42.551199 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:42.551131 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n84x4_7c5930f2-ccc2-476a-8d31-9e85d2735b7a/egress-router-binary-copy/0.log" Jul 29 15:32:42.575400 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:42.575381 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n84x4_7c5930f2-ccc2-476a-8d31-9e85d2735b7a/cni-plugins/0.log" Jul 29 15:32:42.595828 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:42.595804 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n84x4_7c5930f2-ccc2-476a-8d31-9e85d2735b7a/bond-cni-plugin/0.log" Jul 29 15:32:42.623365 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:42.623340 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n84x4_7c5930f2-ccc2-476a-8d31-9e85d2735b7a/routeoverride-cni/0.log" Jul 29 15:32:42.642522 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:42.642501 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n84x4_7c5930f2-ccc2-476a-8d31-9e85d2735b7a/whereabouts-cni-bincopy/0.log" Jul 29 15:32:42.662888 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:42.662855 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-n84x4_7c5930f2-ccc2-476a-8d31-9e85d2735b7a/whereabouts-cni/0.log" Jul 29 15:32:42.976313 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:42.976240 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-jrs97_a609730b-db3d-4c8e-ae4a-c4b0e1eeb57d/kube-multus/0.log" Jul 29 15:32:43.106066 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:43.106038 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fmd7q_1e27a282-f2d3-4aa9-81d3-468c1da0ab8e/network-metrics-daemon/0.log" Jul 29 15:32:43.122318 ip-10-0-132-42 kubenswrapper[2580]: I0729 15:32:43.122295 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fmd7q_1e27a282-f2d3-4aa9-81d3-468c1da0ab8e/kube-rbac-proxy/0.log"